design of wireless emg and its use in the classification of finger
TRANSCRIPT
DESIGN OF WIRELESS EMG AND ITS USE IN THE CLASSIFICATION
OF FINGER MOVEMENTS
_______________
A Thesis
Presented to the
Faculty of
San Diego State University
_______________
In Partial Fulfillment
of the Requirements for the Degree
Master of Science
in
Electrical Engineering
_______________
by
Anju Dalal
Fall 2012
iii
Copyright © 2012
by
Anju Dalal
All Rights Reserved
iv
DEDICATION
This thesis is dedicated to my parents and friends for their inspiration and support.
v
ABSTRACT OF THE THESIS
Design of Wireless EMG and its Use in the Classification of Finger Movements
by Anju Dalal
Master of Science in Electrical Engineering San Diego State University, 2012
This thesis presents design and implementation of a wireless EMG sensor, an algorithm is developed to classify finger positions into open or closed position. The design of wireless EMG sensor utilizes a custom built single channel, EMG signal amplification and conditioning circuit. This signal conditioning circuit converts the EMG signal with ~2mVpp to a 3.3Vpp signal and also filters any noise present in the signal. The signal is then digitized using an Arduino Fio board and wirelessly transmitted using an XBee transreceiver operating in the 2.4 GHz ISM band. The received signal is processed using a python program to detect the onset. Hierarchical Temporal Memory (HTM) is used for classification of the EMG signals. Implementation of classification is done using the NUPIC v1.7 toolset from Numenta Inc. A two layer HTM network is designed to consist of 4x2 nodes. It is trained using an EMG sample data set consisting of 200 samples each for closed and opened position of fingers respectively. The real time movement of a finger is captured by onset detection algorithm and fed to the HTM network for classification. The accuracy of the network was observed to be largely dependent on the size of training EMG sample vector, the number of nodes, number of layers in the network, and maximum groups allowed inside each node in the network. The network was tested with 50 samples each of open and close movement and the accuracy was observed to be 94%.
vi
TABLE OF CONTENTS
PAGE
ABSTRACT ...............................................................................................................................v
LIST OF TABLES ................................................................................................................... ix
LIST OF FIGURES ...................................................................................................................x
ACKNOWLEDGEMENTS .................................................................................................... xii
CHAPTER
1 INTRODUCTION .........................................................................................................1
1.1 Design and Development of EMG Hardware ....................................................3
1.2 Processing and Wireless Transmission of ENG Signals....................................3
1.3 Classification of EMG Signal Using Hierarchical Memory Technique ............3
2 PHYSIOLOGY OF EMG ..............................................................................................5
2.1 Introduction of the EMG Signal ........................................................................5
2.1.1 Characteristics of EMG Signal .................................................................6
2.1.2 Uses and Benefits of EMG Signal ............................................................6
2.2 Physiology of Muscles .......................................................................................7
2.3 Generation of the EMG Signal ..........................................................................8
3 EMG ELECTRODES AND ELECTRODE PLACEMENT .......................................11
3.1 EMG Electrodes ...............................................................................................11
3.2 Factors Affecting SEMG .................................................................................12
3.3 Electrode Placement.........................................................................................14
3.3.1 Anatomy of Human Forearm ..................................................................14
3.3.2 Optimal Position for Electrode Placement ..............................................16
3.3.3 Procedure for Electrode Placement .........................................................17
4 EMG AMPLIFIER DESIGN .......................................................................................19
4.1 EMG Amplifier Circuit ....................................................................................19
4.1.1 Preamplifier.............................................................................................19
4.1.2 Filters ......................................................................................................21
4.1.3 Amplifier and Bias Adjustment Circuit ..................................................22
vii
5 ANALOG TO DIGITAL CONVERSION (ADC) AND WIRELESS TRANSMISSION OF THE EMG SIGNALS .............................................................25
5.1 Features of FIO Board .....................................................................................25
5.2 ADC Conversion ..............................................................................................26
5.2.1 Serial Baud Rate and ADC Reference ....................................................26
5.2.2 Initialize the Timer ..................................................................................27
5.2.3 ADC Conversion Mode ..........................................................................28
5.3 Wireless Transmission Using XBEE Module..................................................29
6 ONSET DETECTION AND DATA COLLECTION .................................................30
6.1 Onset of Movement..........................................................................................30
6.2 Collection of Data ............................................................................................31
7 CLASSIFICATION OF EMG SIGNAL .....................................................................33
7.1 State of Art .......................................................................................................33
7.1.1 Artificial Neural Network .......................................................................33
7.1.2 Back-Propagation (BP) Based Neural Network .....................................34
7.1.3 Fuzzy Mean Max Neural Network (FMMNN) .......................................35
7.2 Other Methodologies Used in the Classification .............................................35
7.2.1 Hidden Markov Model ............................................................................35
7.2.2 Bayes Network ........................................................................................36
7.3 HTM Introduction ............................................................................................37
7.3.1 Overview .................................................................................................38
7.3.2 Structure ..................................................................................................38
7.4 Function of HTM .............................................................................................39
7.4.1 Learning ..................................................................................................40
7.4.2 Stages of Learning ..................................................................................40
7.4.3 Inference .................................................................................................41
7.5 Structure and Operation of Node .....................................................................41
7.5.1 Structure of Node ....................................................................................41
7.5.2 Operation of Node...................................................................................42
7.6 Operation of Node in Hierarchy ......................................................................45
7.6.1 Concepts of Operation of Node in Hierarchy .........................................46
7.6.2 Learning Inside the Node ........................................................................49
viii
7.6.3 Summary of Operation of Node in Hierarchy .........................................50
8 RESULTS ....................................................................................................................53
9 CONCLUSION ............................................................................................................56
REFERENCES ........................................................................................................................57
ix
LIST OF TABLES
PAGE
Table 5.1. Registers Used in ADC ...........................................................................................26
Table 8.1.Test Results for the Classification ...........................................................................55
x
LIST OF FIGURES
PAGE
Figure 2.1. EMG signal for open and close movement of finger. .............................................6
Figure 2.2. Action potential mechanism. ...................................................................................9
Figure 3.1. Different types of electrodes.. ...............................................................................11
Figure 3.2. Differential amplification of two-input channels of equal amplitude. ..................13
Figure 3.3. Picture showing surface electrode. ........................................................................14
Figure 3.4. Pertinent muscles of the forearm for finger movements.. .....................................15
Figure 3.5. Flexor muscles controlling finger movement.. ......................................................16
Figure 3.6. Optimal position for electrode placement.. ...........................................................17
Figure 3.7. Picture showing the location of electrode. ............................................................18
Figure 4.1. Stages of the amplifier design. ..............................................................................19
Figure 4.2. A schematic of the differential amplifier configuration. .......................................20
Figure 4.3. Design of the EMG preamplifier. ..........................................................................20
Figure 4.4. Design of the EMG filters. ....................................................................................21
Figure 4.5. Design of the EMG amplifier and bias circuit. ......................................................22
Figure 5.1. Flow chart showing analog to digital conversion (ADC). .....................................27
Figure 5.2. ADC multiplexer selection register control. ..........................................................29
Figure 5.3. System data flow diagram. ....................................................................................29
Figure 6.1.Onset detection of EMG signal. .............................................................................31
Figure 6.2. Sample of close finger movement. ........................................................................32
Figure 6.3. Sample of open finger movement. .........................................................................32
Figure 7.1. Network structure of HTM. ...................................................................................39
Figure 7.2. Structure of spatial and temporal pooler. ..............................................................42
Figure 7.3. Spatial pooler algorithm. .......................................................................................43
Figure 7.4. Spatial pooling mechanism for the new input. ......................................................44
Figure 7.5. Temporal pooling mechanism. ..............................................................................45
Figure 7.6. Temporal group formation from coincidences. .....................................................46
Figure 7.7. HTM network structure for training. .....................................................................46
xi
Figure 7.8. Diagram showing the input to the level 1 node. ....................................................47
Figure 7.9. Level 1 node in learning mode. .............................................................................48
Figure 7.10. Level 2 node training mode and level 1 node inference mode. ...........................48
Figure 7.11. Diagram showing complete learning of level 1 and level 2 node and the classifier is also enabled...............................................................................................49
Figure 7.12. Spatial pooling inside node. ................................................................................49
Figure 7.13. Inference of level 1 mode. ...................................................................................50
Figure 7.14. Operation of nodes in hierarchy. .........................................................................51
Figure 7.15. Image showing pooling in level 2. ......................................................................52
Figure 8.1. Amplified EMG signal. .........................................................................................53
Figure 8.2. Onset detection. .....................................................................................................54
Figure 8.3. Image showing closing of finger after classification. ............................................54
Figure 8.4. Image showing opening of finger after classification. ..........................................55
xii
ACKNOWLEDGEMENTS
I would like to thank my thesis advisor, Dr. Yusuf Ozturk for his unconditional
support, guidance, and patience during the entire course of the research and thesis. Without
his help and motivation it would have not been possible for me to accomplish this
achievement. It was a great learning experience. Also I would like to thank Dr. Ashkan
Ashrafi and Dr. Kee S. Moon for their time and confidence in me for agreeing to be members
in my thesis committee.
This study has in part been supported by engineering research center for sensorimotor
neural engineering.
Finally special thanks to my friends for their encouragement, moral support, and love.
1
CHAPTER 1
INTRODUCTION
In recent years, there has been a tremendous interest in introducing intuitive
interfaces that can recognize the user’s body movement and translate them into machine
commands. Various biomedical signals acquired from tissues, organs, and/or cell systems
like the nervous system are proposed as drivers of Human Computer Interface. Examples of
such innovative human computer interfaces includes electroencephalogram (EEG) [1], which
measures and records the electrical activity of a human brain, electrooculogram (EOG) [1],
which is used for measuring the resting potential of retina. One such human computer
interface is Electromyogram (EMG), which is used for evaluating and recording the electrical
activity produced by skeletal muscles.
Electromyogram signal (EMG) is the small electrical impulses that are generated by
muscular activity. The technique of recording and measuring the electrical impulses of
muscles is referred as Electromyography. Surface electromyography (SEMG) is one such
EMG, which is a non – invasive technique for measuring muscle electrical activity that
occurs during muscle contraction and relaxation cycles. Some of the applications of SEMG
include physical rehabilitation, urology, biomechanics, and ergonomics. SEMG is also used
in diverse applications such as operating prosthesis limbs, robotic devices, and input interface
for small devices like mobile phones or PDAs. Using SEMG, signals, which contain vital
information regarding the motion of arm, hand, leg or any region of interest from human
body, can be generated. These signals can be used as input of one’s behavior for video games
and rehabilitation engineering [1, 2]. The muscle activity provided by EMG signals
combined with motion information provided by accelerometer offers a promising HCI
(Human Computer interface).
Wireless sensor networks are an active area of research in computer science and
telecommunication. A wireless sensor network consists of spatially distributed autonomous
sensors to monitor physical or environmental conditions and cooperatively pass the data
through the network to a main location. The recent research and developments in the wireless
2
sensor networks have a great impact on many aspects of medical care. For example, by
outfitting patients with wireless, wearable vital sign sensors, collecting detailed real-time
data on physiological status can be greatly simplified [3].
Pattern classification/recognition is an important technique, which is used to classify
various signals generated by EMG. The feature extraction plays an important role in the
pattern classification and the success of pattern recognition system depends on the
effectiveness and choice of features to represent the EMG signals. Various approaches like
use of temporal features, enveloping parameters [4], amplitude of the first burst, spectral
moments [5], and second order ARMA (Autoregressive Moving Average) coefficients have
been employed for feature extraction in the literature. Several classifiers have also been
experimented with, including back-propagation neural networks, neural networks optimized
with genetic algorithms [6], simplified fuzzy ARTMAP networks [7], and Mahalanobis-
distance based ARTMAP networks [8].
In this research, we aim to develop the wireless EMG sensor for the acquisition of the
EMG signals and implementing the pattern matching classification technique. Most of the
work has been done on the recognition of EMG signals generated by upper arm and gross
hand movement tasks. The research concentrates on generating various signals for specific
finger movements.
Early attempts using pattern matching algorithms for prosthesis control have been
proposed by Finley et al [9], Herbert et al [10] and Graupe and Cline et al [11]. In areas like
biomechanics, rehabilitation of disabled individuals [1], prosthetics and human computer
interface [2], uses pattern matching algorithms for signal classification. A dominant part of
related work uses artificial neuronal network (ANN) based classifiers and support vector
machines (SVM) [12-14] for EMG signal classification. Bayesian classifiers [15, 16], fuzzy
classifiers [17, 18], Gaussian mixtures [19], and hidden Markov models [20] are also used.
This thesis work focuses on three important aspects viz, design and development of
EMG hardware, processing and wireless transmission of EMG signal and classification of
EMG signals using hierarchical temporal memory technique.
3
1.1 DESIGN AND DEVELOPMENT OF EMG HARDWARE The amplitude levels of an EMG signal (peak to peak) are in the range 0 – 10
millivolts [21, 22]. Amplification of the signals is required before processing it. Noise and
artifacts can distort the quality of the EMG signal [23, 24]. EMG hardware should be
designed considering several noise contributors including motion of forearm. The equipment
should be small in size because the equipment will be attached to the forearm during the
EMG recording.
1.2 PROCESSING AND WIRELESS TRANSMISSION OF ENG SIGNALS
EMG data is recorded during close and open activity of fingers. An arduino
development board is used for digitizing the EMG signals and an XBee wireless module is
used for wireless transmission of the data to a processing center.
1.3 CLASSIFICATION OF EMG SIGNAL USING HIERARCHICAL MEMORY TECHNIQUE
Hierarchical temporal memory is a machine learning technique that models based on
the memory prediction theory of the brain function. HTMs technique is very similar to the
neural network or other learning based classification techniques. Hierarchical temporal
memory based networks are best described as a memory system that performs inference and
makes predictions. The inference and predictions are made using a network of processing
units referred as nodes. The inference is performed by leveraging the time continuity in the
signal and spatial coherency of EMG signal pattern. HTM has been successfully used to the
Evaluate EEG Signals by John M. Casarella. It is also used for spoken digit recognition by
J.van Doremalen and Song Identification using Numenta platform for intelligent computing
by Schey.
The thesis is organized in eight chapters as described below:
• Chapter 1: Introduction about the thesis and introductory explanation about the further sections.
• Chapter 2: Physiology of the EMG signals .This chapter discusses the EMG signal origin; the generation of action potentials (AP) and characteristics is also discussed in detail.
4
• Chapter 3: EMG electrodes and electrode placement. This section gives detailed explanation of the science behind the electrodes and description about different types of electrodes. The anatomy of human forearms is discussed to determine the optimum electrode placement and factors which needs to be taken care of.
• Chapter 4: In this chapter, the design and implementation of an EMG amplifier design is provided, after general description of the amplifier each component of the device is elaborated. The components are comprised of an electrode extension, a preamplifier, a filter, and an amplifier with bias adjustment. Each of these components is detailed through circuit, diagrams, key equations and the approach taken in the development of the EMG amplifier including the noise/artifact issue and the bias problem.
• Chapter 5: Analog to digital conversion (ADC) and wireless transmission of the EMG signals. We discusses the specification of the analog to digital converter (A/D) interface, and flowchart explaining the working of ADC and how the signals are sent wirelessly.
• Chapter 6: Methodology to collect real time EMG data and onset detection algorithm. This chapter discusses methodology to capture the EMG data and onset detection algorithm.
• Chapter 7: Classification of EMG signals. The chapter focuses on the HTM technique to classify the EMG signals and the background of other classification technique and give a detailed explanation of the mechanism of the classification technique.
• Chapter 8: Finally, this chapter provides the result.
5
CHAPTER 2
PHYSIOLOGY OF EMG
2.1 INTRODUCTION OF THE EMG SIGNAL Electromyography is a technique for evaluating and recording the electrical activity
produced by skeletal muscles. EMG measures the electrical impulses of muscles during
expansion and contraction. An electromyograph detects the electrical potential generated by
muscle cells when these cells are electrically and/or neurologically activated. The signal
represents the electric potential generated by the ionic flow across the membrane of the
muscle fibers that propagates through the intervening tissues to reach the detection surface of
an electrode located in the region of interest. The muscle fibers, when activated generate a
motor unit action potential (MUAP). The activation from the central nervous system is
repeated continuously for as long as the muscle is required to generate force. This continued
activation generates motor unit action potential (MUAP) trains. These trains from the
concurrently active motor units superimpose to form the EMG signal [23]. As the excitation
from the Central Nervous System increases to generate greater force in the muscle, a greater
number of motor units are activated (or recruited) and the firing rates of all the active motor
units’ increases.
The EMG signal can be recorded or measured with an electrode, and is then displayed
on an oscilloscope, which would then provide information about the ability of the muscle to
respond to nerve stimuli based upon the presence, size and shape of the wave – the resulting
action potential. EMG signal could either be of positive or negative voltage since it is
generated before muscle force is produced and occurs at random intervals as shown in the
Figure 2.1.
As with other electro-physiological signals, an EMG signal is small and susceptible to
noise interference, and needs to be amplified with an amplifier that is specifically designed to
measure physiological signals. There are many dependent factors that could affect a surface
EMG and leads to interpretation problems. For example, the contact of electrode to the skin
could distort a recording signal, the inadequate amplification of the signal could cause a
6
Figure 2.1. EMG signal for open and close movement of finger.
recorder detection problem and a wrong filter could affect the desirable information of EMG
signal [23, 25]. Moreover, there are other factors such as the distance between electrodes as
well as the recording times used in the experiment. The device utilized in the measuring of
the signal must also be considered since low-level input into a recording device could also
affect data and yield inaccurate results.
2.1.1 Characteristics of EMG Signal The EMG signal frequency lies between 0 to 500 Hz with the amplitude ranging
between 0 to 10 milli volts (peak-to-peak) or 0 to 1.5 milli volts (rms). However, the usable
energy of EMG signal is dominant between 50-250 Hz [24, 26]. The extended study of EMG
signal characteristics can be found at [21] and [22].
EMG signals are complicated and are affected by the anatomical and physiological
properties of muscles and the control scheme of the nervous system, as well as the
characteristics of the instrumentation used to detect and observe it [23].
2.1.2 Uses and Benefits of EMG Signal Besides basic physiological and biomechanical studies, kinesiological EMG is
established as an evaluation tool for applied research, physiotherapy/rehabilitation, sports
training, and interactions of the human body to industrial products and work conditions [2,
25].
The wide spread uses of EMG can be listed as:
• Medical Research
7
• Orthopedic
• Surgery
• Functional Neurology
• Gait & Posture Analysis
• Rehabilitation
• Post surgery/accident
• Neurological Rehabilitation
• Physical Therapy
• Active Training Therapy
• Typical benefits of EMG
• EMG allows to directly “look” into the muscle
• It allows measurement of muscular performance
• Helps in decision making both before/after surgery
• Helps patients to “find” and train their muscles
2.2 PHYSIOLOGY OF MUSCLES The muscles cause our body parts to move through contraction and relaxation. Each
muscle consists of many muscle fibers which range in length from a few millimeters (mm) to
about 30 centimeters and have a diameter of between 10 and 100 micrometers [27, 28]. The
muscle fibers will shorten to about 57% of their resting length during contraction [27]. All of
the muscle fibers making up a particular muscle do not contract simultaneously but rather
small groups of muscle fibers contract and relax together, as directed by their motor unit.
Motor unit, the most important part in EMG signal, consists of a motor neuron, its axon, and
all of the muscle fibers that it controls [29]. Countless motor units provide the pathway from
the central nervous system to muscle fibers [29]. Just prior to muscle contraction and the
application of force, an increasing number of motor units are recruited for muscle activation
[30]. The motor units are typically recruited according to their size, beginning with the
smaller motor units first. When muscles contract the muscles fibers excite, the excitability of
muscle fibers through neural control represents a major factor in muscle physiology. This
phenomenon can be explained by a model of a semi-permeable membrane describing the
electrical properties of the sarcolemna (muscle membrane).There is exchange of ions across
8
the muscle fibers innervated by the recruited motor units results in small electrical currents
[25, 30], this electric signal is transmitted to its intended target by traveling through an axon,
which combined for a particular motor unit is referred to as the motor unit action potential
(MUAP) [29]. The aggregate electric signal generated from all of the MUAPs in a detected
area is referred to as the myoelectric signal, a recording of which is called an
Electromyogram (EMG) [30]. Note that this signal propagation precedes muscle movement
by around 10 to 100 ms [30]. This 100-ms head start helps in minimizing the time period of
the EMG signal used to classify the finger movements.
2.3 GENERATION OF THE EMG SIGNAL The electric signal which is generated when muscles are excited are transmitted to its
intended target by traveling through an axon. However, an axon is a relatively poor
conductor because it rapidly attenuates the electrical signal. The potential can decrease to 37
% of its original value after traveling a distance of only 0.15 millimeters along an axon,
resulting in an unusable potential value [31]. This distance in which the potential becomes
unusable is called the length constant. The length constant is dependent upon the size of the
axon, as it is proportional to the square root of an axon diameter. To overcome this tendency
of signal attenuation, the nervous system uses a method to increase the strength of the electric
signal. When the potential decreases to a threshold level, such as eight millivolts, the neuron
will fire another 100 millivolts action potential [31].
However, the action potential will keep decreasing after travel through the axon,
which in effect will stimulate the neuron to fire one burst of action potential after action
potential, a process that is referred to as frequency modulation. For example, in order to
make a potential increase to 10 millivolts, the neuron might fire ten times per second,
although the neuron is also able to extinguish voltage in order to end action potential.
Figure 2.2 [32] shows the mechanism of action potential steps:
1. Under normal conditions, the resting cell membrane is more permeable to K+ than to the Ca2+ and Na+ molecules so there is less concentration of Ca2+ and Na+ in extracellular fluid. As a stimulus is received by the dendrites of a nerve cell, this causes the Na+ channel to open if the opening is sufficient to drive the interior potential from -70mV up to -55mV, the process is continuous.
9
Figure 2.2. Action potential mechanism. Source: J. A. Tuszynski and J. M. Dixon, Biomedical Applications of Introductory Physics. New York: Wiley, 2002.
2. Having reached the action threshold, more Na+ channels i.e. voltage –gated channels open. The Na+ influx drives the interior of the cell membrane up to about +30 mV. The process to this point is called depolarization.
3. The Na+ channels close and the K+ channels open. Since the K+ channels are much slower to open, the depolarization has time to be completed. Having both Na+ and K+ channels open at the same time would drive the system toward neutrality and prevent the creation of the action potential.
4. With the K+ channels open, the membrane begins to repolarize back toward its resting potential.
5. The repolarization typically overshoots the rest potential to about -90 mV. This is called hyperpolarization and would seem to be counterproductive, but it is actually important in the transmission of information. Hyperpolarization prevents the neuron from receiving another stimulus during this time, or at least raises the threshold for any new stimulus. Part of the importance of hyperpolarization is in preventing any stimulus already sent up an axon from triggering another action potential in the opposite direction. In other words, hyperpolarization assures that the signal is proceeding in one direction.
6. After hyperpolarization, the Na+/K+ pump eventually brings the membrane back to its resting state of -70 mV.
Consequently, any undesired departure from a perfectly ordered system may give rise
to so called noise. Higher receptor potential will initiate less noisy action potentials.
However, a noisy system is not always bad, as it enables living things to be able to adjust
themselves to changing environment [31].
10
From a practical point of view, one can assume that in a healthy muscle any form of
muscle contraction is accompanied by the described mechanisms. The EMG signal is based
upon action potentials at the muscle fiber membrane resulting from depolarization and
repolarization processes as described above.
11
CHAPTER 3
EMG ELECTRODES AND ELECTRODE
PLACEMENT
3.1 EMG ELECTRODES The EMG electrodes record the bioelectrical events. There are different kinds of
electrodes capable of receiving EMG signals such as wire, needle, and surface electrodes as
shown in Figure 3.1 [33]. The wire and needle electrodes are inserted through the skin into
the muscle tissue while the surface electrodes are placed on the surface of the skin. Wire and
needle electrode are useful for accessing the individual motor unit and deep muscles, and
requires trained personnel.
Figure 3.1. Different types of electrodes. Source: S. Day. (2002, Oct. 18). Important factors in surface EMG measurement [Online]. Available: http://www.bortec.ca/pages/resources.htm.
In this thesis we have used surface electrodes as they are extremely low risk to the
subject which can be applied without pain and are inexpensive. They are the most suitable
electrodes for assessing the time-force relationship of EMG signals and interfacing an
individual with a biomechanical device [33].
There are many surface electrodes available based on the variety of shapes, sizes, and
materials such as monopolar electrode, bipolar electrode, tripolar electrode, multipolar
electrode, barrier or patch electrode, and belly tendon electrode.
12
Monopolar electrode configurations use one electrode to monitor the EMG signals
from the muscle and another electrode as a reference, located on electrically unrelated tissue.
This configuration suffers from its lack of ability to filter unwanted electrical signals from
outside noise sources and muscles surrounding the target muscle [33]. The bipolar electrodes
overcome this shortfall, moreover, it combines accuracy without the invasiveness that other
types of larger electrodes have such as tripolar and multipolar electrodes. Furthermore, with
the bipolar recording technique we can reduce the noise in the EMG signal, so we have
chosen the bipolar electrodes to use in this thesis.
The bipolar electrode employs differential amplification technique, which measures
the voltage difference between two specific points. It has two contacts that are not connected
to each other. Therefore, one node will be used for positive input, and the other will be used
for a negative input for the differential amplifier. Because the differential amplifier treats
both inputs equally, it will yield an accurate output.
In Figure 3.2 [29], differential amplification of two-input channels is explained. The
differential amplifier causes an 180˚ phase reversal of one of the inputs [29]. Any signal that
is common to both input leads of the amplifier i.e. the common mode signal, is thus canceled
out as equal but opposite and does not appear at the output. In this way, correlated sources of
noise and other unwanted signals are mostly excluded and the signal to noise ratio is
significantly increased. The remaining signal is then amplified. However the distance
between the electrodes could affect the measurement result. Placing the electrodes too far
from one another could yield a weak signal. On the other hand, placing them too close may
also result in unusable data, since the amplifier preprocesses each inputs signal separately
before subtracting those signals for output. In addition, there must be another contact used as
a reference point for these two inputs.
3.2 FACTORS AFFECTING SEMG Several factors given below are considered for sEMG (Surface Electromyography) to
improve the quality of EMG signal [21, 33]:
• Electrode shape
• Electrode size
13
Figure 3.2. Differential amplification of two-input channels of equal amplitude. (a) Shows the two channels, with channel 2 inverted. (b) Shows the result of subtracting channel 2 from channel 1, the signal has been completely suppressed. Source: W. W. Campbell, Essentials of Electrodiagnostic Medicine. Baltimore, MD: Williams & Wilkins, 1999.
• Inter-electrode distance
• Electrode material
Electrode Shape: The electrode shape is defined as the shape of the conductive area.
Electrodes come in several shapes such as circular, square and bar shaped electrodes. We are
using a square cloth electrode that has an adhesive surface to make the proper electrode
contact with skin [33]. The electrode pad has dimension of 0.875 inches. The practical
implication is that the surface area for each electrode site (i.e. bipolar) should be the same, so
the input impedance at each site is similar and common mode disturbance is limited.
Electrode Size: The electrode size is defined as the size of the surface of a conductive
area of electrode. The size of the electrodes should not exceed 10 mm or about 0.4 inches
[33] when placed in the direction of the target muscles because upon an increase of the size
in the direction of the muscle fibers, it has an integrative effect on the sEMG signal,
increasing the detected amplitude and decreasing the high frequency contents.
14
Inter-Electrode Distance: Inter-electrode distance is defined as the center to center
distance between the conductive areas of electrodes. Decreasing the size of the conductive
area reduces the effective sEMG measurement distance (i.e. depth). Similarly, decreasing the
inter-electrode distance decreases the effective recording distance and shifts the EMG
bandwidth to higher frequencies. The inter-electrode distance should also not go beyond 1
inch [33, 34] as shown in Figure 3.3.
Figure 3.3. Picture showing surface electrode.
Electrode Material: We are using pre-gelled Ag/AgCl electrodes as they are best
electrodes recommended to capture EMG signal because of the following properties [33]:
1. The Ag/AgCl electrode material forms the very good contact layer with the skin.
2. It maintains low electrode-skin impedance
3. It has very stable behavior in time (that is, with respect to impedance and chemical reactions at the skin interface).
3.3 ELECTRODE PLACEMENT For the optimal electrode placement and to achieve the good and stable surface EMG
signal we should understand the basic anatomy of the forearm as we are collecting the signals
from the forearm muscle fibers controlling the finger and hand movements.
3.3.1 Anatomy of Human Forearm The forearm is divided into two muscles group:
1. Posterior group-Extensors of the wrist and fingers and the supinator.
2. Anterior group – Flexors of the wrist and fingers and the pronators.
The extensor muscles, located on the posterior side of the forearm, are responsible for
extending the fingers outward as initially when the movement starts. The flexor muscles,
15
generally located on the anterior side of the forearm, are responsible for flexion of the finger
joints.
In Figure 3.4 [35] the specific extensor muscles involved are the extensor digitorum,
extensor pollicis longus, extensor pollicis brevis, extensor indicis, and extensor digiti minimi.
The function of each muscle is described below:
1. The extensor digitorum is responsible for the extension of the wrist and finger joints, controlling primarily the middle and ring fingers, and to a lesser extent the index and little fingers.
2. The extensor pollicis longus and extensor pollicis brevis are both responsible for the extension of the thumb and abduction of the wrist.
3. The extensor indicis is responsible for the extension and adduction of the index finger.
4. The extensor digiti minimi is responsible for the extension of the little finger.
The extensor indicis is in a deeper muscle layer of the arm and covered by the extensor
digitorum and extensor digiti minimi muscles.
Figure 3.4. Pertinent muscles of the forearm for finger movements. Source: C. J. Miller, “Real-time feature extraction and classification of prehensile EMG,” M.S. thesis, Dept. Comp. Sci., San Diego State Univ., 2008.
The approximate locations of these muscles are indicated in Figure 6. The extensor
indicis muscles are actually in the muscle layer below the extensor digiti minimi and extensor
digitorum. Naturally, the deeper location of this muscle will have a negative impact on the
energy that surface electromyography can detect. The approximate locations of these muscles
are indicated in Figure 3.4.
16
In Figure 3.5 [35], the specific flexor muscles of interest are the flexor digitorum
superficialis, flexor digitorum profundus, and flexor pollicis longus are shown. The flexor
digitorum superficialis and flexor digitorum profundus are both responsible for flexion of the
four fingers and the wrist, while the flexor pollicis longus controls flexion of the joints of the
thumb. With this basic understanding of the human anatomy, we are better prepared to locate
the target muscles and determine the optimal electrode placement.
Figure 3.5. Flexor muscles controlling finger movement. Source: C. J. Miller, “Real-time feature extraction and classification of prehensile EMG,” M.S. thesis, Dept. Comp. Sci., San Diego State Univ., 2008.
3.3.2 Optimal Position for Electrode Placement Figure 3.6 [33] shows the optimal position of the electrodes. The placement of the
electrode can be explained on the following principles:
• It is parallel to the muscle fibers in order to maximize the probability of reading the same signal.
• Midway between motor point and belly of the muscle.
• Electrodes should not be place on the outer edge of the muscle as other muscles could interfere with the muscle under examination.
17
Figure 3.6. Optimal position for electrode placement. Source: S. Day. (2002, Oct. 18). Important factors in surface EMG measurement [Online]. Available: http://www.bortec.ca/pages/resources.htm.
• Distance should be approximately 2 cm apart.
• Ground electrode should be placed on the bone (electrically neutral area).
3.3.3 Procedure for Electrode Placement The following procedures outline the steps necessary for optimal electrode placement:
1. The electrode skin contact should be as clean as possible. Shaving of forearm hair is preferable, but not necessary. It is suggested that the skin areas be cleaned with alcohol and allowed to dry before proceeding.
2. Properly orient the bipolar electrode. The imaginary line connecting the two electrodes constitutes the axis of orientation. This imaginary line should be in parallel with the muscle group.
3. Position one bipolar electrode correctly oriented on each one of the flexor digitorum profundus muscles shown in Figure 3.7.
4. Position one monopolar electrode on the bone of the wrist or elbow bone. The monopolar electrode serves as reference and has the longest cable length as shown in Figure 3.7.
5. Ensure each electrode is tightly secured. Flexion of the arm should not result in sliding or movement of the electrodes. We have used the Velcro strap to tighten the electrode in order to keep the electrode placement consistent for the duration of the runs. Moreover the electrodes are covered in a mild adhesive which provides further stability.
18
Figure 3.7. Picture showing the location of electrode.
It is important that all electrode placements remain consistent, as much as possible,
for a particular subject and even among multiple subjects. Consistent electrode placement
will reduce training time and classification errors.
19
CHAPTER 4
EMG AMPLIFIER DESIGN
4.1 EMG AMPLIFIER CIRCUIT As shown in the Figure 4.1 the circuit is divided into 4 stages:
1. Preamplifier
2. Filters
3. Amplifier
4. Bias circuit.
TINA TI [36] is used as a tool to design the EMG amplifier circuit schematic. It is a powerful
circuit design and simulation tool. TINA-TI provides all the conventional DC, transient and
frequency domain analysis of SPICE. TINA-TI is ideal for designing, testing and
troubleshooting the basic and advanced circuits.
Figure 4.1. Stages of the amplifier design.
4.1.1 Preamplifier The bipolar electrodes are used with a differential amplifier, as the raw EMG signal
collected by the electrodes are small so the large gain is required and for this purpose we are
using differential amplifier. The differential amplification subtracts the potential at one
electrode from that of the other electrode and then amplifies the difference it is possible to
measure the full effective bandwidth of the EMG signal as shown in the Figure 4.2 [24].
As shown in Figure 4.3, the burr-brown INA 2128 instrumentation amplifier is
selected for this design due to its high CMRR at high gain. The CMRR of the INA2128 at a
gain of 1000 is about 130 dB which is sufficient for our design.
20
Figure 4.2. A schematic of the differential amplifier configuration. The EMG signal is represented by 'm' and the noise signals by 'n'. Source: C. J. De Luca. (2002). Surface electromyography: Detection and recording [Online]. Available: http://www.delsys.com/library/ papers/SEMGintro.pdf.
Figure 4.3. Design of the EMG preamplifier.
21
To calculate the resistance for a certain gain, Equation (4.1) can be used:
Where, R = R1 = R2:
5012AkGRΩ
= + (4.1)
For the first amplification stage a gain of around 1000 is desired:
50 25999 2
kR Ω= = Ω
× (4.2)
The monopolar electrode which serves as the body reference node gives an added
measure of stability while correcting the signal level. The design of the body reference circuit
is taken from the Burr-Brown INA129P instrumentation amplifier datasheet.
4.1.2 Filters A Band pass filter is used after the output of the preamplifier, to eliminate the
frequency component that is outside the EMG signal frequency range as shown in the Figure
4.4. Since the energy of the EMG signal is between frequency range of 0 to 500 Hz and
usable energy within the range of 50-250 hertz, typical band pass frequency ranges are
between 12Hz (high pass filtering) to 500 Hz (low-pass filtering). High-pass filtering is
necessary because movement artifacts and DC components are comprised of low frequency
components (typically <10Hz). Low pass filtering is desirable to remove high-frequency
components to avoid signal aliasing.
Figure 4.4. Design of the EMG filters.
The cutoff frequency of the RC filter can be computed from Equation 4.3 [37].
fCutOff = (4.3)
22
To get a cutoff frequency of 12 hertz a resistor value of 91 kilo ohms, and capacitor
value of 150 nano farads were selected to connect an RC high pass filter. The cutoff
frequency of high pass filter can be computed from Equation 4.4.
fCutOff =
= 11.66 12 Hz (4.4)
To get a cutoff frequency of 500 hertz a resistor value of 3 kiloohms, and capacitor value of
0.1 micro farads were selected to connect an RC low pass filter. The cutoff frequency of low
pass filter can be computed from Equation 4.5.
fCutOff =
= 530 500 Hz (4.5)
4.1.3 Amplifier and Bias Adjustment Circuit The input of the amplifier is the output from the filter. In EMG amplifier as shown in
the Figure 4.5, the BURR-BROWN, OPA228 chip is used to construct a non-inverting
amplifier circuit. The OPA228 chip is a dual op-amp; therefore one op-amp is used for gain
circuit and another op-amp is used for bias adjustment circuit. The amplifier provides ability
to adjust or correct the output signals. The peak to peak amplitude of amplified output signals
should lie around 3.3V as the max voltage handled by Arduino FIO board is 3.3V.
Figure 4.5. Design of the EMG amplifier and bias circuit.
23
Potentiometer R11 is used to tune the amplifier gain. When R11 is zero gain is one as
explained in Equation 4.6 and when R11 is max at 200 kilo ohms the gain equal to 21 as
shown in Equation 4.7.
At R11 = 0:
Gain =
=
= 1 (4.6)
At R11 = 200k:
Gain =
=
= 21 (4.7)
An arduino Fio board is used for A2D conversion. Arduino can measure only positive
voltages. A constant offset has to be added to the signal. This is done by the bias adjustment
circuit that can adjust the signal up or down by the level of positive nine volts, or negative
nine volts as shown in Equation 4.8, 4.9 and 4.10. Potentiometer R13 is used to tune the bias.
Consider VR13 to be the voltage across the potentiometer. Then the Vout from the bias circuit
is given by Equation 4.11 [37].
When:
R13 = 0, VR13 = +Vcc,
= (4.8)
When:
R13 = R13/2, VR13 = 0,
= (4.9)
When:
R13 = R13, VR13 = -Vcc,
= (4.10)
(4.11)
24
Gain is shown in Equation 4.12:
Gain =
+Vcc < VR13 < -Vcc (4.12)
Input voltages to arduino board must stay between GND and AVCC (Reference
voltage 3.3.V). Bias circuit is adjusted to provide an offset of 1.6V keeping the peak to peak
amplitude of amplified EMG signal to be 3.3V max.
25
CHAPTER 5
ANALOG TO DIGITAL CONVERSION (ADC) AND
WIRELESS TRANSMISSION OF THE EMG
SIGNALS
Arduino FIO board is used to convert analog EMG signal to a digital signal (A/D
conversion) before it can be processed and analyzed. The resolution of arduino board is 10
bit. Very small signals may need a higher amplification to achieve a better amplitude
resolution. For EMG almost all of the signal power is located between 50 and 250 Hz .This
would result in a sampling frequency of at least 1500 Hz.
We have used Arduino FIO board for ADC conversion in this thesis as this fits the
requirements. The ADC converts an analog input voltage to a 10-bit digital value through
successive approximation technique.
5.1 FEATURES OF FIO BOARD General features:
• Micro controller: ATmega328P
• Operating Voltage: 3.3V
• Input Voltage: 3.35 -12 V
• Clock Speed: 8 MHz
ADC features:
• 10-bit Resolution
• 13 - 260 μs Conversion Time
• Up to 76.9 kSPS (Up to 15 kSPS at Maximum Resolution)
• 6 Multiplexed Single Ended Input Channels
• 0 - VCC ADC Input Voltage Range
• Free Running or Single Conversion Mode
• Interrupt on ADC Conversion Complete
26
ADC subsystem of the ATmega328P microcontroller contains six (6) registers as
shown in Table 5.1, only three of them are used by the Arduino analog Function for ADC
conversion which is described.
Table 5.1. Registers Used in ADC Address Name Bit7 Bit6 Bit5 Bit4 Bit3 Bit2 Bit1 Bit0
(0x7E) ADMUX REFS1 REFS0 ADLAR - MUX3 MUX2 MUX1 MUX0
(0x7A) ADCSRA ADEN ADSC ADATE ADIF ADIE ADSP2 ADSP1 ADSP0
(0x7B) ADCSRB - ACME - - - ADTS2 ADTS1 ADTS0
(0x79) ADCH ADC Data Register High byte
(0x78) ADCL ADC Data Register Low byte
• ADMUX: It helps in selecting the input channel by writing to the MUX bits in ADMUX. Any of the ADC input pins, as well as GND and a fixed band gap voltage reference, can be selected as single ended inputs to the ADC.
• ADCSRA (ADC status and control registers): The ADC is enabled by setting the ADC Enable bit, ADEN in ADCSRA shown in Table 5.1.
• ADCSRB (ADC status and control registers): The trigger source is selected by setting the ADC Trigger Select bits ADTS.
• ADCH/ADCL (ADC data registers): The ADC generates a 10-bit result after completion of conversion which is presented in the ADC Data Registers, ADCH: ADCL register pair.
5.2 ADC CONVERSION The Figure 5.1 shows the flowchart of ADC. It explains the ADC conversion of the
EMG signals. The baud rate is set to 57600 which is the maximum provided by Arduino Fio
board.
5.2.1 Serial Baud Rate and ADC Reference EMG output from the EMG amplifier circuit is connected to analog input zero pin of
Arduino board. The reference voltage for the ADC (VREF) indicates the conversion range
for the ADC, the minimum value of the 10-bit output of the ADC register (0x000) represents
AGND and the maximum value (0x3FF) represents the voltage on the AVCC
.
The baudrate is set to 57,600 which is the limit for Fio board in the ADC. In our
thesis the default setting for an analog read call within the Arduino IDE with REFS1 and
27
Start
Enable serial baudrateSet ADC reference to VCC
Initilize the Timer to attain 1.5KHz sampling frequency.
Enable ADC Single Conversion Mode and global interrupt
Interrupt subroutine reads the ADC 10 bit value from ADCH:ADCL
registers and sends data wirelessly to base station.
Figure 5.1. Flow chart showing analog to digital conversion (ADC).
REFS0 bits in Multiplexer select (ADMUX) register to REFS1 = 0, REFS0 = 1 is retained
which sets the VREF to VCC which is 3.3V for Fio board.
5.2.2 Initialize the Timer Timer can be set to trigger an interrupt at a certain point in future. Timers run
asynchronously or independently from the main program. Timers work by incrementing a
counter variable, also known as a counter register, until it reaches its maximum value and
counter overflows, or the value in the timer matches the compare match register, at which
point the timer resets. The timer normally sets a flag bit to indicate an overflow or compare
match and this triggers an interrupt. An Interrupt Service Routine (ISR) specific to the
interrupt is executed when the timer overflows. The ISR will reset the overflow flag.
28
ATmega328 has 3 timers called timer0, timer1 and timer2. Timer0 and timer2 are 8bit timer,
where as timer1 is a 16bit timer.
Clear Timer on Compare Match mode is used to interrupt and achieve 1.5 KHz ADC
frequency. In order to achieve the above we have to load the timer1and compare register with
count that will hit 1500 times a second:
• Timer should fire every 1/1500th of a second or 0.67 milli second.
• Timer frequency is same as that of Fio board clock frequency of 8MHz.
• Timer Firing Period = Timer Resolution * (Number of Timer Counts + 1).
In CTC mode, when the timer matches our desired count it will reset itself to zero.
This takes one clock cycle to perform, so we need to add a factor of +1 into our calculations:
• (Number of Timer Counts + 1) = (Timer Firing Period) / (Timer Resolution)
• (Number of Timer Counts + 1) = (0.67 * 10-3) / (0.125 * 10-6)
• (Number of Timer Counts + 1) = 833
• Number of Timer Counts = 832
We have to load the timer1 compare register with 832 to attain a frequency of 1.5 KHz.
5.2.3 ADC Conversion Mode There are two modes of operation:
• Free running Mode
• Single conversion Mode
Free Running Mode: The feature of free running mode is that ADC is set up to start a
new conversion immediately after the previous conversion is complete. So in Free-Running
mode the ADATE bit is set to one and trigger select bit ADTS [2:0] in ADCSRB are set to
zero refer to Table 5.1. This selects the ADC interrupt flag ADIF as the trigger source. The
ADC now operates in Free Running mode, constantly sampling and updating the ADC Data
Registers.
Single Conversion Mode: We have used Single Conversion mode to achieve the
desired conversion rate in our thesis. In this mode the ADC must be enabled before starting
the conversion by setting the bit ADEN = 1 in the ADCSRA Register. Now the conversion is
initiated manually, by setting the ADSC bit in the ADCSR Register. We have used Timer
Interrupt to control the frequency of conversion.
29
The first conversion is started by writing a logical one to the ADSC bit in ADCSRA
(refer to Figure 5.2). At the beginning when the Arduino boot loader initializes the Analog
subsystem of the ATmega328P micro controller, the ADC ADMUX register is set to
0100_0000 as shown in the Figure 5.2. By setting the REFS0 to one we are setting the ADC
reference voltage to be 3.3V.
Figure 5.2. ADC multiplexer selection register control.
By default, the analog read function result is presented right adjusted (ADLAR = 0)
because the ADC generates a 10-bit result which is stored in the ADC Data Registers,
ADCH: ADCL register pair.
5.3 WIRELESS TRANSMISSION USING XBEE MODULE XBee and XBee-PRO Modules are low-cost, low-power wireless transreceivers
engineered to meet Zigbee/IEEE 802.15.4 [38]. The modules require minimal power and
provide reliable delivery of critical data between devices. Figure 5.3 shows the data flow
diagram in UART interfaced environment [38]. Data enters the RF module through the DIN
Pin, the data is stored in the serial receive buffer until it is processed.
Figure 5.3. System data flow diagram.
Interrupt subroutine reads the ADC 10 bit value from ADCH: ADCL registers and
sends data wirelessly to base station via XBee module.
30
CHAPTER 6
ONSET DETECTION AND DATA COLLECTION
6.1 ONSET OF MOVEMENT The first step for data collection is the proper identification of the onset of movement.
This is very important because early estimation results in low level signals from rest position
being captured while delayed estimation can leave out the most important initial MUAP’s
[39]. The success of the eventual classifier depends upon the correct part of the EMG signal
extracted for analysis. We have used threshold method for detecting the onset in our thesis.
The threshold based onset detection, Hodges and Bui detection algorithm is a finite
moving average (FMA) algorithm [39]. FMA algorithms use a fixed-size sliding window to
scan the EMG signal, averaging the signal within the window and comparing it to a baseline
window. When the current window exceeds the baseline by a predetermined number of
standard deviations, the algorithm halts, returning the first index of the current window as the
estimated onset time [39].
We have modified Hodges and Bui detection algorithm for onset detection in our
thesis. By analysis of the EMG signal we calculated and inferred that the standard deviation
of the signals is very high when the movement starts. We exploit this feature of EMG signals
to determine the onset of the movement. We adopt the sliding window concept from Hodges
and Bui algorithm. The standard deviation of 3 seconds samples without the movement is
captured which is considered as baseline standard deviation of signal without movement. The
threshold standard deviation of baseline signal (tb) is calculated to be a fixed multiple of
baseline standard deviation calculated without the movement.
EMG samples are read serially in real time and stored. Once the number of samples
read equals the window size, standard deviation of the window is calculated (stdw). If stdw
exceeds the tb we assume the onset has started and we store the index of EMG sample
collected. Only if n out of m successive windows exceed tb and m does not exceed another
threshold of number of samples, Tm, designating the minimum time period to be considered
31
active [39] we confirm the onset has occurred. Since the sampling frequency of EMG signal
is 1.5 KHz and the active signals of movement lies within 300ms to 400ms [30] thus we read
512 samples of data from the start of onset for classification. Figure 6.1 shows the raw EMG
data with onset detected data.
Figure 6.1.Onset detection of EMG signal.
6.2 COLLECTION OF DATA EMG signals amplified from the EMG circuit are converted to digital form by
arduino FIO board and transmitted wirelessly using XBee. The sampled digital data is
collected through another XBee board at the base station which sends the received data
serially and this is collected by python program running on the computer.
Python program reads the incoming real time data serially and performs the onset
detection discussed earlier in Section 6.1. We have collected samples based on this approach
and signal reproduction is close to the one seen on oscilloscope during the movement of
fingers.
Figure 6.2 shows snapshots of EMG signals of Close finger and Figure 6.3 shows the
snapshots of Open finger captured via this approach.
In both Figures 6.2 and 6.3 Y axis shows the digital value of converted EMG signals
using the 10 bit ADC and range between 0 – 210 (1023), X-axis represents the time in ms of
collected EMG samples.
32
Figure 6.2. Sample of close finger movement.
Figure 6.3. Sample of open finger movement.
33
CHAPTER 7
CLASSIFICATION OF EMG SIGNAL
7.1 STATE OF ART EMG data acquired from Arduino board is a sequence of digital data samples. It is
observed that the EMG signal generated is unique for open and close movements of fingers.
This property is used for classification. There are a wide variety of classifiers that already
exists for this purpose. These classifiers are mainly used in Human Computer Interaction
(HCI) systems [40]. Some of the important techniques that were examined and application to
HCI found during the course of our research are discussed in the following sections.
7.1.1 Artificial Neural Network Neural network is a computing technique that evolved from mathematical models of
neurons and systems of neurons. In 1993, Putnam and Knapp proposed AR model parameters
based feature vector neural network for pattern recognition technique for a real-time
computer control system of the EMG from user’s gestures [41]. The techniques used were
Single Layer Perception (SLP) structure and Multi-Layer Perception (MLP) structure. The
SLP was trained by Widrow Hoff LMS algorithm whereas a back propagation algorithm was
used to train Multi-Layer Perception (MLP) structure. With the feature vector comprise with
AR model parameters 95% accuracy was achieved in classification utilizing both biceps and
triceps data. It is considered more robust classifier required for persons with disabilities.
Another prominent attempt based on one layer feed-forward neural network
technique is purposed by Rosenberg, and utilized in EMG controlled 2-dimensional pointer
which is known as Biofeedback Pointer [42]. This graphic input device is controlled by wrist
motion. The pointer detects the EMG signals of four of the muscles used to move the wrist.
These are interpreted by a neural network which is trained for each user. The Biofeedback
Pointer’s simple neural network is computationally inexpensive, but with the side effect of a
reduction in accuracy which is compensated for by using four EMG sensors. During training
period, the network calculated 8 times with offset 0 to 448ms for finding out least error
network. The reason behind this is to minimize the reaction time delay regarding user’s
34
motion. The performance yields 14% according to Fitt’s law. This technique is not much
efficient as main problem with the current training is that the user’s motions may not
adequately synchronize with the cursor therefore more sophisticated neural network and
better training methods required for future improvement.
In 2006 G. Tsenov et al. implemented concept of the feature extraction to improve the
classification performance of hand and finger movements [43]. The main key is to improve
the classification accuracy up to 98% is to use the both time and frequency domain features
of the EMG signal for the identification of forearm activity. In the time domain, features like:
mean absolute value (MAV), variance (VAR), waveform length (WL), norm, number of zero
crossings, absolute maximum, absolute minimum, maximum minus minimum and median
value (Med) are extracted.
After a year later Kyung Kwon Jung et al. came with stronger classifiers that would
help to implement the HCI [44]. They proposed a Yule-Waler algorithm of pattern
recognition of EMG signals of hand gesture based on AR model for spectral estimation and
LVQ (Learning quantization vector. The algorithm calculates power spectral density (PSD)
of the EMG signals and LVQ is a method for training competitive layers in a supervised
manner using the 4th order AR model parameters [44]. A competitive layer automatically
learns to classify input vectors and based AR model for spectral estimation. However, the
classes that the competitive layer finds are dependent only on the distance between input
vectors. There is no mechanism in a strictly competitive layer design to say whether or not
any two input vectors are in the same class or different classes. The experiment verified that
EMG signals produced by hand gestures are reliably classified by proposed system with a
success rate of about 78%.
7.1.2 Back-Propagation (BP) Based Neural Network Naik, Kumar, and Weghorn implemented BPN to overcome the drawback of the
standard Artificial Neural Network (ANN) architecture by augmenting the input hidden
context units, which give feedback to the hidden layer, thus gives the network an ability of
extracting features of the data from the training events [45]. Four RMS EMG values were the
inputs to the ANN. The outputs of the ANN were the different isometric hand action RMS
35
values. The overall accuracy was reported as 97%, but the number of hand gesture
identification was restricted to three.
In 2008, Naik, Kumar, and Palaniswami proposed ICA (Independent component
analysis) based signal extraction method used to improve identification of various hand
gestures using multi run ICA of SEMG with back propagation learning algorithm based
ANN classifier [46]. They reported that only ICA is not suitable for SEMG due to the nature
of SEMG distribution and order ambiguity [46]. The Temporal de-correlation source
separation (TDSEP) algorithm based ICA gives overall classification accuracy of 99% for all
the experiments and can be used for the classification of different subtle hand gestures.
However, BPN cannot realize high learning and discrimination performance because the
EMG patterns differ considerably at the start and end of the motion even if they are within
the same class.
7.1.3 Fuzzy Mean Max Neural Network (FMMNN) In 2004, Kim, Jeong, and Son applied fuzzy mean max neural network (FMMNN) as
a classifier for online EMG mouse that controls computer cursor [47]. The stochastic values
such as integral absolute value were used as features for an appropriate classification of the 6
intended wrist motions like left, right, up, down, click and rest operation. Difference
Absolute Mean Value (DAMV) extracted from the 4 channel EMG signals is used as the
input vectors in learning and classifying the patterns. The Pattern recognition rate for each
wrist motion reported is above 90% whereas average recognition rate yield 97% .Therefore it
can be used as an efficient means of HCI.
7.2 OTHER METHODOLOGIES USED IN THE CLASSIFICATION
There are several methodologies used for classification, few methods are explained
below
7.2.1 Hidden Markov Model The HMM has an inherent ability to deal with spurious misclassification. During
classifier training, HMM provides large computational savings compared to MLP. HMM
yields satisfactory recognition results. Wheeler applied HMM devices as the pattern
36
recognition method for the designing and using neuroelectric interfaces for controlling virtual
devices [48]. He used Moving average feature among the most common method like Short
Time Fourier Transform (STFT), Wavelets, Moving Average, Auto-Regression (AR) [48].
Hand gestures are used to interface with a computer instead of manipulating mechanical
devices such as joysticks and keyboards. EMG signals are non-invasively sensed from the
muscles used to perform these gestures. These signals are then interpreted and translated into
useful computer commands. Further improvement required in model correcting adaptation
and calibration stage to decrease error rates depending on sleeve position, sweating, skin
moisture, length of time that electrodes were worn and fatigue. The achieved classification
accuracy with HMM is 87% [1].
Although the HMM are perfectly suitable algorithms for the classification of time
series and very popular dynamic classifiers in the field of speech recognition, they are not
much popular within the HCI community.
7.2.2 Bayes Network In 2007, Xiang Chen et al. implemented multiple hand gesture recognition along with
a 2-D accelerometer for mobile HCI [49]. The structured type movements have higher
classification success rate than pointing movements. The Feature extraction like common
time domain and frequency domain features is carried out to reduce the data dimensionality
while preserving the signal patterns which help to differentiate between the gesture classes.
In their research, MAV, the ratio of two MAVs, and fourth-order AR model coefficients are
used in the formation of the feature vectors [49]. The accelerometer feature vector consists of
the mean absolute values. The Linear Bayesian Classifier is trained with the feature vectors
to distinguish the different gesture actions from each other. Due to their low computational
complexity and stable recognition performance; classical linear classifiers are well suited for
real-time gesture analysis and real life implementation. It was reported that the combination
of accelerometers and SEMG sensors provided increased classification accuracy rate by 5-10
% [1].
In 2008, Kim, Mastnik, and André implemented the combination of two simple linear
classifiers K-nearest neighbor (KNN) and Bayes for development of an EMG based interface
for hand gesture recognition [50]. The K-nearest neighbor (KNN) classifier and the Bayes
37
classifier are comparatively simple and thus fast algorithms. The algorithms generally
provide proportionally good results therefore best suited for the real time system. The KNN
classifier, which belongs to the non-parametric statistical classifiers, rates a pattern by
regarding the most similar labeled training samples. For this purpose, the distances (e.g.
Euclidean distance) between the feature vector of the current pattern and the feature vectors
of each training sample are calculated. Beforehand, all vectors are generally normalized. The
number of adjacent samples which are taken into account is defined by the parameter k. The
presented EMG-based control interface is able to reliably recognize various hand gestures
with a positive classification rate of over 94% even though only one single EMG sensor is
used, in contrast to related work which is based on multiple EMG sensors [1]. The feature
selection is important for better classification and increasing number of features does not
always produce good result. Therefore small discrepancies can result major differences in
EMG signal as well as degrade the performance of classifier.
Consequently, there is a wide range of potential applications using EMG signal in
human-machine interfacing. However, to realize advanced applications, many issues still
need to be resolved, including the development of algorithms for EMG-specific analysis, the
extraction of relevant features, and the design of real-time classifiers with guaranteed
accuracy.
7.3 HTM INTRODUCTION HTM (Hierarchical temporal memory) technique is very similar to the neural network
or other learning based classification techniques. It is a technique of learning patterns that is
being developed by Numenta Inc. The technique developed by Numenta is available for
academic research and has been used by other researchers around the world for different
pattern recognition and learning purposes. The tool used for the HTM technique is called
NuPIC (Numenta Platform for Intelligent Computing), is a software development platform
that provides tools to create, train, test, and deploy HTMs [51]. The thesis builds upon
algorithms given in the (NuPIC) version 1.7.
HTM technique uses a network of nodes which are trained initially to classify the
incoming data accurately. The trained network can then be run in inference mode which can
be used to classify any random incoming.
38
HTM technique uses the following main characteristics [52]:
• A hierarchy in space and time to share and transfer learning.
• Slowness of time, which, combined with the hierarchy, enables efficient learning of intermediate levels of the hierarchy.
• Learning of causes by using time continuity and actions.
• Models of attention and specific memories.
• A probabilistic model specified in terms of relations between a hierarchy of causes.
• Belief Propagation in the hierarchy to use temporal and spatial context for inference.
7.3.1 Overview The Hierarchical Temporal Memory (HTM) is a model that replicates the structural
and algorithmic properties of the neocortex, it is the part of the brain which is capable of
performing tasks such as visual pattern recognition, understanding spoken language, and
recognizing and manipulating objects by touch [37]. The HTM network can be considered a
form of Bayesian network as there are many similarities where the network consists of a
collection of nodes arranged in a tree-shaped hierarchy [53]. Node acts as a classifier that can
be trained in an either supervised or unsupervised manner. During training, it requires a
temporally structured stream of input data while inference is done independently for every
frame. The heart of an HTM network is hierarchical structuring of the data in both, space
and time which is crucial for the algorithm to succeed [54]. HTM is fundamentally a memory
based system, it trained with time varying data, and relies on storing a large set of patterns
and sequences.
HTMs can be viewed as a type of neural network. However, on its own, the term
“neural network” is not very useful because it has been applied to a large variety of systems.
HTMs model neurons (called cells when referring to HTM), which are arranged in columns,
in layers, in regions, and in a hierarchy. For the more details regard HTMs reader can refer
material provided in reference [53, 54].
7.3.2 Structure A HTM network is organized as a tree-shaped hierarchy. The network consists of
nodes where, each node in the hierarchy self-discovers a set of causes in its input through a
process of finding common spatial patterns and then finding common temporal patterns.
39
Figure 7.1 shows the HTM network structure used in our thesis .The network
structure is organized in a 2 level hierarchy which has a classifier node, a sensor and a
category sensor, where the sensor receives EMG dataset input from CSV vector file, and
category sensor receives category data input for the HTM network respectively. The whole
input data is distributed amongst the leaves of the tree. After an initial processing, the data
results are passed upwards to its parent. This process repeats until the information reaches the
top of the tree. The root is also called classifier node because it will output the result of the
inference and also, it receives the categorical information during the process of supervised
learning. HTM structure as well as memory is hierarchical in both space and time. The
benefit of hierarchical organization is that it significantly reduces training time and memory
usage because patterns learned at each level of the hierarchy are reused when combined in
novel ways at higher levels.
Figure 7.1. Network structure of HTM.
7.4 FUNCTION OF HTM This section discusses about the main function of HTM. The Hierarchical temporal
memory based networks are best described as a memory system that performs inference and
makes predictions. The inference and predictions are drawn using a network of processing
units referred to as nodes which will be explained in detail further on in Section 7.5. The
40
inference is performed by leveraging the time continuity in the signal and spatial coherency
of EMG signal pattern.
7.4.1 Learning An HTM region learns about its world by finding patterns and then sequences of
patterns in sensory data. The region does not “know” what its inputs represent; it works in a
purely statistical realm. It looks for combinations of input patterns that often occur together,
which we call spatial patterns. It then looks for how these spatial patterns appear in sequence
over time, which we call temporal patterns or sequences [54].
7.4.2 Stages of Learning The HTM network has 2 distinct stages of learning which are given below:
1. Training
2. Inference
The algorithm of training and inference are based on two principles for each node which are
given below:
• Understanding the operations within a single node.
• Understanding how a node operates with other nodes in the hierarchy.
Training: Training of the network happens layer-wise. The layers are numbered
starting with the leaves as shown in Figure 7.1 in Section 7.3.2. Once the first layer is trained,
it switches into inference mode, thus producing an output for the next layer to be trained on.
This is repeated until the classifier-node is reached.
After initial training, an HTM can continue to learn or, alternatively, learning can be
disabled after the training phase. Another option is to turn off learning only at the lowest
levels of the hierarchy but continue to learn at the higher levels. Once an HTM has learned
the basic statistical structure of its world, most of the new learning occurs in the upper levels
of the hierarchy. If an HTM is exposed to new patterns that have previously unseen low-level
structure, it will take longer for the HTM to learn these new patterns [38]. If Categorical
information is available, it can also be trained in a supervised way but in that case the
classifier node is the only node to be trained in a supervised manner.
41
7.4.3 Inference After an HTM has learned the patterns in its world, it can perform inference on novel
inputs. When an HTM receives input, it will match it to previously learned spatial and
temporal patterns. Successfully matching new inputs to previously stored sequences is the
essence of inference and pattern matching. Inference in an HTM region is similar. HTM
region must handle novel input during inference and training.
The mechanism behind the inference is that during inference when HTM receives any
input, every node starts by determining the distance of the new input to every coincidence. In
the second step, this vector of distances is turned into a “belief distribution” by calculating
the probability that the input belongs to each coincidence utilizing a Gaussian distribution.
Note that the result is, in general, not a probability distribution because it does not always
sum to one. This vector is then used to calculate a belief distribution over all temporal
groups. The belief for a temporal group is determined by simply taking the maximum belief
of all coincidences belonging to that group. The resulting vector is then passed upwards to
the node’s parents [38, 53].
Training and inference will be covered in more detail with respect to node in Section
7.6 in detail.
7.5 STRUCTURE AND OPERATION OF NODE In this section, the detailed explanation of the node structure and operation of the
node will be given with respect to the designed HTM network used in this thesis.
7.5.1 Structure of Node A node is the processing units of HTM network, it consists of two modules to
perform the training and inference functions.
1. Spatial Pooler: The spatial pooler takes advantage of spatial coherency in data; it learns a mapping from a potentially infinite number of input patterns to a finite number of quantization centers. The output of the spatial pooler is in terms of its quantization centers.
2. Temporal Pooler: The temporal pooler takes advantage of the temporal continuity in the incoming data; it makes temporal groups according to the temporal proximity of occurrence of the quantization centers of the spatial pooler. The output of the temporal pooler is in terms of the temporal groups that it has learned.
42
The Figure 7.2 shows structure of a node. The input to the node is connected to the
input of the spatial pooler. The output of the spatial pooler is the input to temporal pooler.
The output of the temporal pooler is the output of the node. The node has an input pattern
size of 5. The spatial pooler within this node has 4 quantization centers marked C1 to C4.The
output of the spatial pooler is a vector of length 4. The temporal pooler within this node has 2
groups marked g1 and g2 in Figure 7.2. The output of the temporal pooler, and hence of the
node, is a vector of size 2.
Figure 7.2. Structure of spatial and temporal pooler.
7.5.2 Operation of Node Each node in the hierarchy is exposed to a temporal stream of data and, independent
of its position, performs the same two tasks: finding reoccurring spatial patterns and common
sequences of those patterns. These two operations are called spatial pooling and temporal
pooling.
Spatial Pooling: During training, the node performs a matching of the input data to
the already selected data to reduce possibly infinite number of different inputs to a small set
of selected data. The small sets of selected data are called coincidences. For the nodes at the
bottom level of the hierarchy, the input comes directly from the sensor. For higher-level
nodes, the inputs are the outputs from their child nodes. For any given parent node, its input
is a concatenation of the outputs from its children. The input to the level 2 node is a
concatenation of these outputs from two child nodes.
43
The Spatial Pooler algorithm takes in a maximum distance parameter maxDistance
(Euclidean distance) which corresponds to the minimum separation at which a pattern is
considered different from the existing quantization patterns as shown in Figure 7.3. The
Euclidean distance of the incoming EMG signal is calculated from the existing quantization
center patterns. The quantization center with similar pattern as incoming signal has minimum
Euclidean distance. As shown in Figure 7.3 the minimum Euclidean distance of 10 between
the incoming EMG signal and existing quantization shows the similarity in pattern.
Figure 7.3. Spatial pooler algorithm.
The Euclidean Distance can be calculated from the Equation 7.1:
ED=
Where:
EMG = (X1, X2, X3………. Xn) and Quantization Center = (Y1, Y2, Y3…….. Yn). (7.1)
The maxDistance parameter sets the maximum Euclidean distance at which two input
vectors are considered the same during learning. For every input pattern, it is checked
whether there is a quantization center that is within Euclidean distance maxDistance if a
quantization center exist then nothing is done if there is no quantization center within
44
maxDistance a new quantization centers is created and added to the list of quantization
centers [55].
Figure 7.4, explains the mechanism for the EMG input, showing the new coincidence
C9 which will be adding up in the spatial pooler memory. Also we can infer when the
condition satisfies there is no new coincidence in the spatial memory pooler.
C5C4 C6
C7 C8
C2C1 C3SPATIAL POOLER
C5C4 C6
C7 C8
C2C1 C3SPATIAL POOLER
C9
C5C4 C6
C7 C8
C2C1 C3SPATIAL POOLER
C9
SPATIAL POOLER MEMORY
SPATIAL POOLER MEMORY
INPUT VECTOR
INPUT VECTOR
SPATIAL POOLER MEMORY
New Coincidence is formed
Input Vector Euclidian Distance is < MaxDistance so no new Coincidence is
created
Figure 7.4. Spatial pooling mechanism for the new input.
Temporal Pooling: This step is based on the idea that patterns which frequently
transit to one another are likely to belong to the same “cause”. Therefore, treating them in the
same way can allow useful invariants to be formed. This is implemented by maintaining a
transition matrix T, with every element Tij corresponding to the number of transitions from
the i-th to the j-th coincidence . This matrix is then normalized forming a Markov chain that
contains estimated probabilities for each transition as shown in Figure 7.5.
45
3 2
1 2
1 1
0.3 0.2
0.1 2
0.1 1
0.0
0.0
0.0
CI
C2
C3
CI C2 C3Transition Matrix
NormalizedTransition Matrix
Time
0
0
0
C1
C1
C1
C2
C3
C3
C3
C2
C2
C1
C1
Figure 7.5. Temporal pooling mechanism.
In the next step, the probability from the normalized transition matrix is used as a
measure of similarity. Then these coincidences are merged using a Hierarchical
Agglomerative Clustering algorithm and the grouping continues until a pre-defined number
of groups (maxGroups) are reached. Those resulting sets of coincidences are called temporal
groups. As shown in the Figure 7.6 the temporal group 1 and temporal group 2 are formed.
The temporal group 1 consist the coincidences C2 temporal group 2 consists of C1, C3
coincidences respectively based on the probabilities.
7.6 OPERATION OF NODE IN HIERARCHY This section will cover the detailed concepts of operation of nodes in hierarchy in
HTM network. Also after the discussion of concepts, the simplified two level hierarchical
network is shown in Figure 7.14 (p. 51).
46
C2
0.20.1
TEMPORAL GROUP 2
TEMPORAL GROUP 1
0.1
0.2
C1
C3
0.3
0.1
Figure 7.6. Temporal group formation from coincidences.
7.6.1 Concepts of Operation of Node in Hierarchy All the nodes are trained level by level in HTM network. Figure 7.7 illustrates an
HTM network used for classification of EMG signals. The network consists of 8 nodes at
level1, 2 nodes at level2 and a classifier node which uses the data from level2 node to
perform classification. The input vector length is 512 EMG samples and the classifier used in
this case is a standard K-Nearest Neighbor (KNN) classifier available in the NUPIC toolset.
Figure 7.7. HTM network structure for training.
47
The benefit of hierarchical organization is efficiency. It significantly reduces training
time and memory usage because patterns learned at each level of the hierarchy are reused
when combined at higher levels. The hierarchy enables a new object in the world to inherit
the known properties of its sub-components. To learn a new high level object you don’t have
to relearn its components.
There is a tradeoff between how much memory is allocated to each level and how
many levels are needed. Fortunately, HTMs automatically learn the best possible
representations at each level given the statistics of the input and the amount of resources
allocated. If you allocate more memory to a level, that level will form representations that are
larger and more complex, which in turn means fewer hierarchical levels, may be necessary. If
you allocate less memory, a level will form representations that are smaller and simpler,
which in turn means more hierarchical levels may be needed. In this study a 2 level
hierarchy model of HTM is used to reduce the training and inference time [51]. During the
training of HTM network, level 1 node receives input from a CSV file that contains the EMG
data. The input EMG vector length has 512 EMG samples. Since there are 8 nodes in the first
level which receive this EMG vector, each node will get an input of vector length 64 as
shown in Figure 7.8.
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
1 2 3 4 5 6 87INPUT
VECTOR
LEVEL 1
Figure 7.8. Diagram showing the input to the level 1 node.
After getting the input to the first level node, all the level-1 nodes in the hierarchy are
trained. During this time all the level-2 nodes are disabled and since level1 nodes are in
learning mode they don’t send any outputs to the level-2 nodes as shown in Figure 7.9. (The
disabled nodes are shown in gray color.)
48
Level 2
Level 1
DisabledTEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Classifier
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Train
Category Data
Figure 7.9. Level 1 node in learning mode.
Once the level-1 nodes finish their learning process, the nodes at the second level are
enabled and placed into learning mode. The nodes in the level-1 are switched from learning
mode to inference mode. This process is shown in Figure 7.10. (Note that Classifier is still
disabled).
Level 2 Node
Level 1 Node
Disabled
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Classifier
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Train
Category Data
Infer
Figure 7.10. Level 2 node training mode and level 1 node inference mode.
After their training, the level 2 nodes will be in inference mode and the whole training
process repeats continuously until classifier node is reached.
As the category data is fed, the level 1 and level 2 nodes completes their learning and
both are placed in inference mode as shown in Figure 7.11.
49
Level 2
Level 1
InferenceTEMPORAL
POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Classifier
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Inference
Category Data
Figure 7.11. Diagram showing complete learning of level 1 and level 2 node and the classifier is also enabled.
7.6.2 Learning Inside the Node The learning inside a node also takes place hierarchically. Thus the Spatial pooling
inside a node is performed before the temporal pooling is performed as shown in Figure 7.12.
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Disabled
Disabled
Disabled
Train
Level 1 Node
Level 2 Node
Figure 7.12. Spatial pooling inside node.
Once the spatial pooling is completed, the spatial pooler inside the node is switched
to inference mode from training mode. The temporal pooler is enabled and set to learning
mode as shown in Figure 7.13.
50
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
Train
Disabled
Disabled
Inference
Level 1 Node
Level 2 Node
Figure 7.13. Inference of level 1 mode.
7.6.3 Summary of Operation of Node in Hierarchy This section will summarize the concept explained in previous Sections 7.6.1 and
7.6.2 and illustrates the complete operation of node in hierarchy with the simplified 2 level
hierarchal structure as shown in Figure 7.14. The level 1 consist of 4 nodes and level 2
consist of 2 node. The input to these nodes is an EMG data sample of vector length 256
which is divided amongst each node as shown in Figure 7.8 earlier.
Each bottom-level node has finished learning and are in inference mode. The level-1
nodes have learned 16 quantization centers each (C1 to C16) by the process of spatial
pooling. The quantization centers with their corresponding spatial pooler and node outputs
are shown in the Figure 7.14.The input pattern to the node 1(N11) of level matches
quantization center C2 which is shown in red color, within the spatial pooler of this node.
For the second node (N21) of the level1 the input pattern to this node matches quantization
center C3. This is reflected in the output of the spatial pooler, with a one in position 3(in red
color) and zeros elsewhere. Similarly input to the node (N31) and node (N41) matches
quantization center C1 and C5 respectively as illustrated in Figure 7.14.
After training of the spatial pooler, training of the temporal pooling will start using
the algorithm described in the section for temporal pooling. The node (N0, N1, N3 and N4)
in level1 is formed with 7 temporal groups and each temporal group containing 2
quantization centers.
During inference mode of temporal pooler, each co-incidence input to the first level
node is matched with using the Euclidean distance to the existing set of co-incidences. The
51
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
C1
C5 C6
SPATIAL POOLERC2 C3 C4
C7 C8
C9 C10 C11 C12
C13 C14 C15 C16
Group 0
TEMPORAL POOLER
Group 1
Group 2
Group 3
Group 4
Group 5
Group 6
Group 7
Group 0Group 1Group 2Group 3
Group 4
Group 5
Group 6
Group 7
C1 C2 C3
C5
C4
C6 C7 C8
C9 C10 C11 C12
C13 C14 C15
TEMPORAL POOLER
SPATIAL POOLER
C16
C1
C5 C6
SPATIAL POOLERC2 C3 C4
C7 C8
C9 C10 C11 C12
C13 C14 C15 C16
Group 0
TEMPORAL POOLER
Group 1
Group 2
Group 3
Group 4
Group 5
Group 6
Group 7
Group 0Group 1Group 2Group 3
Group 4
Group 5
Group 6
Group 7
C1 C2 C3
C5
C4
C6 C7 C8
C9 C10 C11 C12
C13 C14 C15
TEMPORAL POOLER
SPATIAL POOLER
C16
INPUT
Level 1
Level 2
k1
N11 N21 N31 N41
N22N21
Figure 7.14. Operation of nodes in hierarchy.
co-incidence in spatial pooler’s database that matches the current input co-incidence is
presented to the temporal pooler in level 1 node. Since there are 7 groups in the temporal
pooler the first level nodes, output of the temporal pooler in level1 nodes is a binary vector of
length 7, with each bit set to 0/1. The bit in binary vector identifying the group,
corresponding to the co-incidence presented by the spatial pooler is now set as ‘1’ and the
bits corresponding to the other groups are set as ‘0’.
Learning process for 2 level nodes (N21 and N22) is started after completion of the
learning process of the level1 nodes. Level2 nodes are now enabled and the level1 nodes are
placed in inference mode when training the level2 nodes. Nodes present in second level of
52
the network are of the same type as that present in the first level. Hence the learning and
training process followed by nodes in the second level are the same as that in the first level.
Input to a level2 node is a concatenation of the inputs from its child nodes. Since input from
one child node is of length 7 and there are 2 child nodes per node in the second level, the
input is represented as a vector of length 14.
This input is obtained by concatenating the outputs of the bottom-level nodes and this
can be reflected in the output of spatial pooler as, red in position 1 and position 13. In the
second level node’s (N22) spatial pooler has already learned one quantization center before
the beginning of the input sequence. The new input sequence introduced one additional
quantization center to the second level node as shown in Figure 7.14. A snapshot of a
possible case of spatial and temporal pooler database generated inside second level nodes is
shown in Figure 7.15.
N22 N22
TEMPORAL POOLER
SPATIAL POOLER
TEMPORAL POOLER
SPATIAL POOLER
e e
C1
C2
C3
C4
C5
C1
C2
C3
C4
C5
G1
G2
G3
G1
G2
Figure 7.15. Image showing pooling in level 2.
53
CHAPTER 8
RESULTS
In this research EMG amplifier circuit is successfully designed to capture the EMG
signals from flexor digitorum profundus, a muscle responsible for finger movement from the
open and close movement of fingers. Noise, motion artifacts and DC biasing problems were
solved with proper design of circuit and using the low noise SMD components for final
assembly on PCB. Figure 8.1 shows the amplified EMG signal.
Figure 8.1. Amplified EMG signal.
Arduino FIO board A/D converter is used to sample the analog EMG signals at 1.5
KHz sampling rate. The sampling rate can also be varied using the timers in Arduino board.
The sampled EMG signals was transmitted using the XBEE module attached to the FIO
board. FIO board successfully served the dual purpose of sampling the signals and
transmitting them.
The transmitted signal is captured by another XBEE module attached to the
computer. The Matlab program reads the data from the XBEE module serially. The GUI is
developed to display the raw EMG data and data after onset is detected and the
corresponding image of open and close finger respectively after classification.
Onset Detection, Classification and Observations: The collection of EMG signals is
referenced for a window of 3 seconds (3000 samples) during which the fingers are opened and
54
closed repeatedly. With the onset detection function the onset of the movement is detected
successfully, if there is a movement made as shown in Figure 8.2.
Figure 8.2. Onset detection.
Training samples are collected and the HTM network is trained with these samples.
After training an XML file with training data and network is created and this is used for
testing the online data. The raw EMG signal and the onset movement are displayed in GUI.
After classification based on open and close movement of finger the image of open and close
are displayed on GUI as show in Figure 8.3 and Figure 8.4 respectively.
Figure 8.3. Image showing closing of finger after classification.
55
Figure 8.4. Image showing opening of finger after classification.
Using the EMG device on right forearm, the classifier is trained with 200 repetitions
of each close and open movement of ring fingers. Real time classification is performed with
100 total movements, which included 50 of each close and open movement, in random order.
The experimental results are shown in Table 8.1. It shows the Actual Movement vs.
Movement classified by HTM classifier.
Table 8.1.Test Results for the Classification
Actual Movement Vs.
Movement classified by HTM
OPEN
CLOSE
OPEN
46
4
CLOSE
2
48
HTM classifier successfully classified open movement 46 times out of 50 actual
movements with accuracy for open movement being 92% and close movement 48 times out
of 50 actual movements with accuracy for open movement being 96%. The overall accuracy
of HTM classifier is 94%.
There were two false positive during close movement and four during open
movement.
56
CHAPTER 9
CONCLUSION
In this thesis we have successfully designed an EMG circuit, collected the EMG
signals from the close and open finger movement, transmitted the captured EMG signal via
Xbee, detected the onset of the movement in real time and classified the movement using
HTM network.
The research has demonstrated a simple way to measure the open and close movement of
fingers and implemented a basic real-time algorithm for continuous onset detection. The future
work involves having feature extraction method capable of mitigating the additional noise
included while travelling through different tissue and change in anatomical and physiological
properties of muscles over period of time. Having a more advanced hardware with built in
preamplifier on electrode mount and wireless module embedded into the electrode that can
significantly reduce the noise and make the unit more portable and easy to configure and use.
57
REFERENCES
[1] M. R. Ahsan et al., “EMG signal classification for human computer interaction: A review,” Euro. J. Sci. Res., vol.33, no.3, pp. 480-501, Jan. 2009.
[2] Y. Makino et a.l, “EMG sensor integration based on two-dimensional communication,” in Proc. 5th Int. Conf. Networked Sensing Systems, Kanazawa, Japan, 2008, pp. 140-147.
[3] I. Noorzaie. (2006). Survey paper on medical applications of wireless networks [Online]. Available: http://www.cse.wustl.edu/~jain/
[4] Y. Hou, “Hand preshaping EMG signal feature extraction and grasp classification for prosthetic hand control,” M.S. thesis, Dept. Math. Stat., San Diego State Univ., CA, 1996.
[5] M. Vuskovic and S. Du, “Spectral moments for feature extraction from temporal signals,” in Proc. Int. Conf. Intelligent Computing, Hefei , China, 2005, pp. 1063-1072.
[6] B. Dundon, “A genetic algorithm in the design of a neural network for EMG pattern recognition,” M.S. thesis, Dept. Comp. Sci., San Diego State Univ., CA, 2002.
[7] M. Vuskovic and S. Du, “Classification of EMG patterns with simplified fuzzy ARTMAP networks,” in Proc. 2001 Int. Joint Conf. on Neural Networks, Washington, DC, 2001, pp. 1-6.
[8] H. Xu, “Mahalanobis distance based ARTMAP networks,” M.S. thesis, Dept. Comp. Sci., San Diego State Univ., CA, 2003.
[9] R. R. Finley and R. W. Wirta, “Myocoder studies of multiple myocoder response,” Arch. Phys. Med. Rehabil., vol. 48, p. 598, Mar. 1967.
[10] P. Herberts, “Myoelectric signals in control of prostheses,” Acta. Orth. Scand., vol. 40, p. 124, Jul. 1969.
[11] J. Graupe and K. Cline, “Functional separation of EMG signals via ARMA identification methods for prosthesis control purposes,” IEEE Trans. Syst., Man, and Cybernetics, vol. 5., pp. 252-259, Mar. 1975.
[12] B. Hudgins et al., “A new strategy for multifunction myoelectric control,” IEEE Trans. Biomed. Eng., vol. 40, no. 1., pp. 82-94, Jan. 1993.
[13] A. Hiraiwa et al., “EMG pattern recognition by neural networks for prosthetic fingers control – Cyber finger,” in Proc. 2nd Int. Symp. Measurement Control Robotics, Tsukuba Science City, Japan, 1992, pp. 535-542.
[14] D. Nishikawa et al., “EMG prosthetic hand controller discriminating ten motions using real-time learning method,” Intel. Robots Syst., vol. 3., pp. 1592-1597, Oct. 1999.
58
[15] K. Englehart et al., “Improving myoelectric signal classification using wavelet packets and principal components analysis,” in Proc. 21st Int. Conf. IEEE Eng. in Med. Biol. Soc., Atlanta, GA, 1999, p. 569.
[16] K. Englehart and B. Hudgins, “A robust, real-time control scheme for multifunction myoelectric control,” IEEE Trans. Biomed. Eng., vol. 50, no. 7, pp. 848-854, Jul. 2003.
[17] B. Karlik et a.l, “A fuzzy clustering neural network architecture for multifunction upper-limb prosthesis,” IEEE Trans. Biomed. Eng., vol. 50, no.11, pp. 1255-1261, Nov. 2003.
[18] F. Chan et al., “Fuzzy EMG classification for prosthesis control,” Rehab. Eng., vol. 8, no. 3, pp. 305-311, Sept. 2000.
[19] Y. Huang et al., “Optimized gaussian mixture models for upper limb motion classification,” Eng. Med. Biol. Soc., vol. 1, pp. 72-75, Sept. 2004.
[20] A. Chan and K. Englehart, “Continuous myoelectric control for powered prostheses using hidden markov models,” IEEE Trans. Biomed. Eng., vol. 52, no. 1, pp. 121-124, Jan. 2005.
[21] C. J. De Luca. (2002, Aug. 13). Fundamental concepts in EMG signal acquisition [Online]. Available: http://www.delsys.com/Attachments_pdf/WP_Sampling1-4.pdf
[22] C. J. De Luca. (1997). The use of surface electromyography in biomechanics [Online]. Available: http://delsys.com/decomp/078.pdf
[23] C. J. De Luca. (2006). Electromyography: Encyclopedia of medical devices and instrumentation [Online]. Available: http://www.delsys.com/KnowledgeCenter/ Tutorials_Technical%20Notes.html
[24] C. J. De Luca. (2002). Surface electromyography: Detection and recording [Online]. Available: http://www.delsys.com/library/papers/SEMGintro.pdf
[25] P. Konrad. (2005 Apr.). The ABC of EMG: A practical introduction to kinesiological electromyography [Online]. Available: http://www.demotu.org/aulas/controle/ ABCofEMG.pdf
[26] G. Li et al.,“Selection of sampling rate for EMG pattern recognition based prosthesis control,” in Proc. 32nd Annu. Int. Conf. IEEE EMBS, Buenos Aires, Argentina, 2010, pp. 5058-5061.
[27] J. V. Basmajian and C. J. De Luca, Muscles Alive. Baltimore, MD: Williams & Wilkins, 1985.
[28] A. C. Guyton, Basic Human Physiology. Philadelphia, PA: W. B. Saunders, 1971.
[29] W. W. Campbell, Essentials of Electrodiagnostic Medicine. Baltimore, MD: Williams & Wilkins, 1999.
[30] B. Gerdle et al., “Acquisition, processing and analysis of the surface electromyogram,” in Modern Techniques in Neuroscience Research, U. Windhorst and H. Johansson, Eds. New York: Springer, 1999, pp. 706-712.
59
[31] S. Deutsch and A. Deutsch, Understanding the Nervous System: An Engineering Perspective. New York: IEEE Press, 1993.
[32] J. A. Tuszynski and J. M. Dixon, Biomedical Applications of Introductory Physics. New York: Wiley, 2002.
[33] S. Day. (2002, Oct. 18). Important factors in surface EMG measurement [Online]. Available: http://www.bortec.ca/pages/resources.htm
[34] D. R. Ankrum, “Questions to ask when interpreting surface electromyography (SEMG) research,” in Proc. Human Factors Ergonomics Soc. Annu. Meeting., San Diego, CA, 2000, pp. 530-533.
[35] C. J. Miller, “Real-time feature extraction and classification of prehensile EMG,” M.S. thesis, Dept. Comp. Sci., San Diego State Univ., CA, 2008.
[36] Texas Instruments. (n.d.). SPICE-based analog simulation program [Online]. Available: http://www.ti.com/tool/tina-ti&DCMP=hpa_amp_general&HQS= Other+OT+tina-ti
[37] D. L. Terrell, Op Amps: Design, Application, and Troubleshooting. Newton, MA: Butterworth Heinemann, 1996.
[38] Digi International. (2010, Mar. 17). 2.1.1. UART-interfaced data flow for XBee-PRO XSC RF modules manual (90000938_B) [Online]. Available: http://www.scribd.com/doc/51750338/5/UART-Interfaced-Data-Flow
[39] G. Staude et al., “Onset detection in surface electromyographic signals: A systematic comparison of methods,” EURASIP J. Appl. Signal Processing, vol. 2, pp. 67-81, Jun. 2001.
[40] G-C. Chang et al., “Real-time implementation of electromyogram pattern recognition as a control command of man-machine interface,” Med. Eng. Phys., vol. 18, no. 7, pp. 529-37, Oct. 1996.
[41] W. Putnam and R. B. Knapp, “Real-time computer control using pattern recognition of the electromyogram,” in Proc. 15th Annu. Int. Conf. IEEE Eng. Med. Biol. Soc., San Diego, CA, 1993, pp. 1236-1237.
[42] R. Rosenberg, “The biofeedback pointer: EMG control of a two dimensional pointer,” in Proc. 2nd Int. Symp. Wearable Computers, Pittsburgh, PA, 1998, pp. 162-163.
[43] G. Tsenov et al., “Neural networks for online classification of hand and finger movements using surface EMG signals,” in Proc. 8th Sem. Neural Network Appl. Elect. Eng., Belgrade, Serbia, 2006, pp. 167-171.
[44] K. K. Jung et al., “EMG pattern classification using spectral estimation and neural network,” in Proc. 2007 Annu. Conf. SICE, Takamatsu, Japan, 2007, pp. 1108-1111.
[45] G. R. Naik et al., “Performance comparison of ICA algorithms for isometric hand gesture identification using surface EMG intelligent sensors,” in Proc. 3rd Int. Conf. Sensor Networks Inform., Melbourne, Australia, 2007, pp. 613-618.
60
[46] G. R. Naik et al., “Multi run ICA and surface EMG based signal processing system for recognizing hand gestures,” in Proc. 8th IEEE Int. Conf. Comput. Inform. Technol., Sydney, Australia, 2008, pp. 700-705.
[47] J-S. Kim et al., “A new means of HCI: EMG-MOUSE,” in Proc. 2004 IEEE Int. Conf. Syst., Man and Cybernetics, The Hague, Netherlands, 2004, pp. 100-104.
[48] K. R. Wheeler, “Device control using gestures sensed from EMG,” in Proc. 2003 IEEE Int. Workshop Soft Computing Industrial Applicat., Binghamton, NY, 2003, pp. 21-26.
[49] X. Chen et al., “Hand gesture recognition research based on surface EMG sensors and 2D-accelerometers,” in Proc. 11th IEEE Int. Symp. Wearable Computers, Boston, MA, 2007, pp. 11-14.
[50] J. Kim et al., “EMG-based hand gesture recognition for real-time biosignal interfacing,” in Proc. 13th Int. Conf. Intel. User Interfaces, Maspalomas, Spain, 2008, pp. 30-39.
[51] Numenta. (2011, Sept. 12). Hierarchical temporal memory including HTM cortical learning algorithms [Online]. Available: http: //www.numenta.com/htm-overview/education/HTM_CorticalLearningAlgorithms.pdf
[52] Numenta. (2007, Mar.). HTM comparison with other models [Online]. Available: http://numenta.com/htm-overview/education/HTM_Comparison.pdf
[53] Numenta. (2006). Hierarchical temporal memory concepts, theory, and terminology [Online]. Available: http://www.numenta.com/ Numenta_HTM_Concepts.pdf
[54] D. George and B. Jaros. (2007, Mar.). The HTM learning algorithms [Online]. Available: http://www.numenta.com/for-developers/education/
[55] Numenta. (2008, Sept.). Advanced NuPIC programming [Online]. Available: http://www.numenta.com/for-developers/software/pdf/nupic_prog_guide.pdf