human computer interface using emg signals - hand gesture based manipulator control

8/11/2019 Human Computer Interface Using Emg Signals - Hand Gesture Based Manipulator Control

1/91

HUMAN COMPUTER INTERFACE USING EMG

SIGNALS: HAND GESTURE BASED MANIPULATOR

CONTROL

A PROJECT REPORT

Submitted by,

CB106EI003 AKSHAY KUMAR

CB106EI007 V J ARUN RAJA

CB106EI009 ASWATH R MAHADEV

CB106EI010 BHARAT BALAGOPAL

CB106EI013 DEEPAK G VISWANATHAN

Under the guidance of Mr. SANJIVI ARUL, Asst. Professor, Department of

Mechanical Engineer ing

I n partial fu lf ilment for the award of the degree

Of

BACHELOR OF TECHNOLOGY

IN

ELECTRONICS AND INSTRUMENTATION ENGINEERING

AMRITA SCHOOL OF ENGINEERING, COIMBATORE


2/91

AMRITA VISHWA VIDYAPEETHAM

COIMBATORE 641 105

APRIL 2010

AMRITA VISHWA VIDYAPEETHAM

AMRITA SCHOOL OF ENGINEERING, COIMBATORE

BONAFIDE CERTIFICATE

This is to certify that the project report entitled HUMAN COMPUTER INTERFACE

USING EMG SIGNALS: HAND GESTURE BASED MANIPULATOR CONTROL

Submitted by,

CB106EI003 AKSHAY KUMAR

CB106EI007 V J ARUN RAJA

CB106EI009 ASWATH R MAHADEV

CB106EI010 BHARAT BALAGOPAL

CB106EI013 DEEPAK G VISWANATHAN

in partial fulfilment of the requirements for the award of the Degree of Bachelor of

Technology in ELECTRONICS AND INSTRUMENTATION ENGINEERING is a

bonafide record of the work carried out under my guidance and supervision at Amrita School

of Engineering, Coimbatore.

Project Advisor Group Coordinator

Mr. Sanjivi Arul Mr. P. V. Sunil Nag

Assistant Professor Assistant Professor (ECE dept)

(Mechanical dept) Instrumentation Design

& Research Group

Chairman ECE

Dr. V. P. Mohandas

The project was evaluated by us on:Internal Examiner External Examiner


3/91

i

ACKNOWLEDGEMENT

Our heartfelt gratitude to our Pro Chancellor Br. Abhayamrita Chaitanya and Vice

Chancellor Dr. P. Venkat Rangan for having provided on the necessary infrastructure

required for the successful completion of our project.

We are thankful to Dr. V. P. Mohandas, Chairperson, Department of Electronics and

Communication Engineering for having given the opportunity to do this project.

We are extremely thankful to our group Co-ordinator Mr. P.V. Sunil Nag, Assistant

Professor, Department of Electronics and Communication Engineering and Prof. R.

Sundararajan, Department of Electronics and Communication Engineering for reviewing

our weekly work and providing valuable suggestions.

We are greatly indebted to express our gratitude to our project guide Mr Sanjivi Arul,

Assistant Professor, Department of Mechanical Engineering for his valuable guidance,

support and encouragement.

We are also greatly indebted to our friends for their invaluable suggestions, for their

assistance during testing of our project and also for their tremendous moral support during

tough times.

Above all we would like to thank AMMAwhose words of inspiration, courage, love, support

and wisdom paved us the way for the successful completion of the project.


4/91

ii

We would li ke to dedicate thi s work to our beloved parents and our project

guide M r. Sanji vi Arul , Assistant Professor,Department of Mechanical

Engineering


5/91

iii

CONTENTS

S.NO TITLE PAGE

NUMBER

ACKNOWLEDGEMENT

ABSTRACT vii

LIST OF FIGURES viii

LIST OF TABLES xi

LIST OF ABBREVIATIONS xii

1 INTRODUCTION 1

1.1 BACKGROUND 3

1.2 OBJECTIVE 3

1.3 APPROACH 3

2 LITERATURE SURVEY 5

3 MYO-SIGNALS AND ITS CHARACTERISTICS 8

3.1 FUNCTIONING OF VOLUNTARY MUSCLE FIBRE 9

3.2 ACTION POTENTIAL OF A CELL 9

3.3 FACTORS AFFECTING MYO-SIGNALS 10

3.4 ELECTROMYOGRAPHY(EMG) 10

3.5 FACTORS AFFECTING EMG 11

4 SIGNAL ACQUISITION AND CONDITIONING 12

4.1 DATA ACQUISITION 13

4.2 ELECTRODES AND ITS CLASSIFICATIONS 14


6/91


7/91

v

4.12 SIGNAL CONDITIONING 23

4.12.1 INSTRUMENTATION AMPLIFIER 23

4.12.2 FILTERS 24

4.12.3 FILTER42 PROGRAM 25

4.12.4 FILTER42 PROGRAM STEPS 25

4.13 CIRCUIT DIAGRAM AND PCB LAYOUT 26

4.14 SOUNDCARD INTERFACE 28

5 SIGNAL PRE-PROCESSING 29

5.1 DIGITAL FILTERS 30

5.2 ADAPTIVE THRESHOLDING 31

5.3 CROPPING OF THE FILTER SIGNAL 31

6 GESTURES AND SIGNALS 32

7 CLASSIFICATION 36

7.1 ROOT MEAN SQUARE VALUE(RMS) 37

7.2 SUPPORT VECTOR MACHINES(SVM) 38

7.2.1 RESULT FOR SVM 40

7.3 K-NEAREST NEIGHBOUR CLASSIFIER 40

7.3.1 RESULT FOR K-NN 42

7.4 ARTIFICIAL NEURAL NETWORK(ANN) 42

7.4.1 RADIAL BASIS FUNCTION 42

7.4.2 RBF ARCHITECTURE 43

7.4.3 ALGORITHM OF RBF 44


8/91

vi

7.4.4 RESULT FOR ANN 44

8 PERFORMANCE ANALYSIS 45

9 MANIPULATOR DESIGN AND CONTROL 47

9.1 THE SERVO MECHANISM 48

9.1.1 RC SERVO 49

9.1.2 CONTROLLING A SERVO MOTOR 49

9.2 MANIPULATOR DESIGN AND CONSTRUCTION 50

9.3 MANIPULATOR CONTROL 51

9.3.1 PIC 16F877APERIPHERALS USED 51

9.4 BASIC CONTROL STRATEGY 52

9.4.1 USART INTERFACE 52

9.4.2 PIC 16F877A-SERVO CONTROL ALGORITHM 54

9.5 MANIPULATOR CONTROL USING HAND GESTURES

RESULTS

55

10 CONCLUSION 58

11 FUTURE SCOPE 60

REFERENCES 62

APPENDIX-A 66

APPENDIX-B.1 75

APPENDIX-B.2 76

APPENDIX-B.3 76

APPENDIX-B.4 77


9/91

vii

ABSTRACT

The art of gesture recognition involves identification and classification of gestures. A gesture

is any reproducible action or a sequence of actions. There are lots of techniques and

algorithms to recognize gestures.

In this project, the gestures are recognized using biological signals generated by the human

body. This leads to an interaction between the human user and the computer. These bio-

signals show a quantitative change in response to a gesture. These changes are then

identified, extracted and classified. This results in the classification and recognition of

gestures. The efficiency and reliability of this method lies in the classifier that is used to

classify the signals. The identification and extraction phases of this technique are very easy

and quick, once the characteristics of the biological signals are known.

There are many biological signals that can be used for gesture recognition. Some of them are

Electroencephalogram (EEG), Electrocardiogram (ECG), and Electromyogram (EMG). EMG

signals are generally used because they have good signal strength (in the order of mV). The

acquisition of EMG signals is easy and less complex when compared to the other signals.

The final part of this project is the re-creation of the gestures that has been recognized. The

re-creation of gestures has lots of applications including prosthetic arms, robotic arm,

manipulators etc. The robotic arm has applications in lots of areas. In industries, the action of

the robotic arm can be magnified many folds to handle precise and heavy duty operations.

The robotic arm can also be used as a prosthetic hand to help the disabled. The system also

forms a benchmark tool to study various classification algorithms.


10/91


11/91

ix

Figure 4.17 UAF42 Implementation of a low pass filter with cut-off at

1000Hz

26

Figure 4.18 Circuit Diagram of One-channel Data Acquisition Board 27

Figure 4.19 PCB Layout of One-channel Data Acquisition Board 27

Figure 4.20 Back Side of the Designed Board 27

Figure 4.21 Front Side of the Designed Board 27

Figure 4.22 Block Diagram of Sound Card Interface 28

Figure 5.1 Digital filter block diagram 30

Figure 5.2 Simulink system for data acquisition 30

Figure 6.1 Strap containing DAQ hardware worn by the user 33

Figure 6.2 Angle measurement apparatus with angles marked 33

Figure 6.3 EMG signals obtained for corresponding gesture 34/35

Figure 7.1 Scatter plot for various movements taken into consideration 37

Figure 7.2 SVM algorithm for efficient classification 39

Figure 7.3 SVM Classifier for Up2 & Up3 Movement 39

Figure 7.4 SVM Classifier for Down & Grab Movement 39

Figure 7.5 SVM Classifier for Down & Up3 Movement 39

Figure 7.6 SVM Classifier for Up3 & Grab Movement 39



Figure 7.9 Diagram depicting KNN for classification 41

Figure 7.10 The algorithm for classification using KNN Classifier 41

Figure 7.11 Basic architecture of RBF classifier 43


12/91


13/91

xi

LIST OF TABLES

Table 7.1 Confusion Matrix for SVM Classifier 40

Table 7.2 Confusion Matrix for KNN Classifier 42

Table 7.3 Confusion Matrix for ANN Classifier 44

Table 8.1 Time required for classification 46

Table 8.2 Classification accuracy of three Classifiers 46


14/91

xii

LIST OF ABBRIEVIATION

1. HCI Human Computer Interface

2. EOG Electro-Oculogram

3.

EEG Electroencephalogram

4.

ECG Electrocardiogram

5. EMG Electromyogram

6.

sEMG Surface Electromyogram

7.

DOF Degrees of Freedom

8. ANN Artificial Neural Networks

9.

K-NN K Nearest Neighbour

10.

SVM Support Vector Machines

11.MUAP Motor Unit Action Potential

12.DB Decibels

13.PCB Printed Circuit Board

14.

RMS Root Mean Square

15.ADC Analog-Digital Converter

16.DAC Digital-Analog Converter

17.

GPIB General Purpose Interface Bus

18.PCI Peripheral Component Interface

19.DIP Dual Inline Package

20.RBF Radial Basis Function

21.DC Direct Current

22.RC SERVO Radio Controlled Servo

23.PWM Pulse Width Modulation

24.

CAD Computer Aided Design25.USART Universal Synchronous Asynchronous Receiver

Transmitter

26.

CCP Compare/Capture/Pulse Width Modulation

27.CRT Cathode Ray Tube

28.SCI Serial Computer Interface

29.EEPROM Electrically Erasable Programmable Read Only Memory

30.TTL Transistor Transistor Logic

31.

USB Universal Serial Bus


15/91

1

CHAPTER 1

INTRODUCTION


16/91

2

Human Computer Interface involves the interaction between humans and computers. Several

techniques are used for implementing HCI. Image processing, speech recognition, touch

sensing, bio-signal processing are some of the popular HCI techniques. In this project, a

gesture recognition system is implemented using bio-signals acquired from the body. The

electrical impulses produced in the body during metabolism are analysed to recognise

specific patterns. Different types of bio-signals like electro-encephalograph (signals from the

brain), electro-oculograph (signals from the eye-socket), and electromyograph (signal

produced when muscles expand or contract) can be used. This work involves the use of

Electromyogram (EMG) signals to implement a gesture recognition system.

The basic goal of HCI is to improve the interactions between users and computers by making

computers more usable and receptive to the user's needs. Specifically, HCI is concerned with:

methodologies and processes for designing interfaces (i.e., given a task and a class of

users, design the best possible interface within given constraints, optimizing for a

desired property such as learning ability or efficiency of use)

methods for implementing interfaces (e.g. software toolkits and libraries; efficient

algorithms)

techniques for evaluating and comparing interfaces

developing new interfaces and interaction techniques

developing descriptive and predictive models and theories of interaction

A long term goal of HCI is to design systems that minimize the barrier between the human's

cognitive model of what they want to accomplish and the computer's understanding of the

user's task. Bio signals such as EEG, EMG can be used for HCI applications. A well

developed system for acquisition and analysis of bio signals will suffice all the needs for HCI

systems.

EMG is the signal produced when a muscle expands or contracts. This appears as a potential

difference on the surface. By tapping the surface we can get an EMG for the particular

gesture or action performed. This signal helps in HCI. Different actions produce different

EMGs. So by analysing and processing the EMG signal we can recognize the gesture or

action performed. A classifier system helps in classifying the EMG signal and categorising it


17/91


18/91

4

Figure 1.1 Overview of the project

Analysis: The feature extraction is done where the proper feature is identified which

differs for all the gesture values. There are classes designed according to the feature

values and the classification is done using any of the classification techniques such as

KNN, SVM, ANN etc. The designed manipulator performs the gesture based on the

class of the action.

A robotic manipulator has been designed that will be controlled by the computer based on the

gesture performed by the user. The signals produced in the muscles of the user will help in

controlling the manipulator. In this project, the gestures are restricted to the ones made by the

wrist and the number of the gestures is six. The gestures made by the user are used to control

the DOF of the manipulator.


19/91

5

CHAPTER 2

LITERATURE SURVEY


20/91

6

Surface electromyography is the technique widely used in Human Computer Interface rather

than invasive techniques due to the practical feasibility issues, which is dealt with later on in

this chapter [1].

De Luca [2] gives a comprehensive idea about the characteristics of myosignals and also

about surface electromyography. The EMG signal has maximum energy in the range of 20 -

250 Hz. Differential Electrode Configuration should be used for surface EMG measurements

in order to reduce the noise associated with the signals. The electrodes used should have

detection surfaces consisting of two parallel bars: each 1.0 cm long, 1-2 mm wide, and 1.0 cm

apart. The ideal location of the electrode is on the midline of the muscle belly, between the

myotendonous junction and the nearest innervation zone, with the detection surface oriented

perpendicularly to the length of the muscle fibres.

The differential amplifier used should have at least the following configurations:

Common mode rejection ratio > 80 dB

Noise < 2 V RMS (20 - 400 Hz)

Input impedance > 100 mega ohms

The signals obtained from the body are to be amplified using a single chip instrumentation

amplifier called AD620. This single ship amplifier satisfies all the criteria specified earlier [3,

10].

A comparative study between the temporal and spectral features shows that the use of

spectral features does not have any advantage in terms of efficiency over the temporal

features. Hence temporal features are used in our project. [4]

Y.Yazama et al [5] gives us an idea about wrist movements that can be recognized using

Artificial Neural Networks (ANN). The wrist movements can be easily classified because

there are distinct muscles involved in the generation of wrist movements. By increasing the

number of channels of EMG input signals, the gestures can be identified with more accuracy.

The Root-Mean-Square (RMS) value of a signal is an appropriate feature because RMS gives

the energy of the signal which can be extracted and does not change much during repeated

measurements. [6]


21/91

7

A combination of simple linear classifiers can be used for real-time classification assuring

acceptable recognition accuracy. The classifiers that can be used are K-NN, Bayesian

classifier, etc. [7]

Artificial Neural Networks can be used for classifying the signals. The most efficient method

in ANN is the Radial Basis Function method. [8]

An articulated robotic arm is built using servo motors. The manipulator is powered using

servo motors because they have an error control feedback mechanism which prevents the arm

from moving because of load change. [9]


22/91

8

CHAPTER 3

MYO SIGNALS AND ITS CHARACTERISTICS


23/91

9

3.1 FUNCTIONING OF A VOLUNTARY MUSCLE FIBER

Figure 3.1 Functioning of a muscle fibre

The signals obtained from the surface of the body are called myosignals. They are mainlyobtained due to the movement of the muscles. There are two movements of the muscles:

contraction and expansion. Skeletal muscles are composed of individual muscle fibres that

contract when stimulated by a motor neuron. Motor neurons originate in the ventral horn of

the spinal cord and consist of a cell body, dendrites and an axon. The axon projects to a

muscle where it branches, forming synapses with muscle fibres. A motor unit is the smallest

functional subdivision of a muscle. It consists of the motor neuron, its axon and all the

muscle fibres that are innervated by its branches. When motor units are activated, the

corresponding muscle fibres contract.

3.2 ACTION POTENTIAL OF A CELL

Every cell in the human body is enclosed by a cell membrane. An action potential is a

transient variation of the membrane potential of a cell membrane of an excitable cell due to

the activity of voltage gated ion channels embedded in the membrane. The course of the

action potential can be divided into five parts: the rising phase, the peak phase, the falling

phase, the undershoot phase, and finally the refractory period. During the rising phase the

membrane potential depolarizes (becomes more positive). The point at which depolarization

stops is called the peak phase. At this stage, the membrane potential reaches a maximum.

This is followed by a falling phase. During this stage the membrane potential hyperpolarizes

(becomes more negative). The undershoot phase is the point during which the membrane

potential becomes temporarily more negatively charged than when at rest. The time during

which a subsequent action potential is impossible to fire is called the refractory period, which

may overlap with the other phases. Muscle action potentials are provoked by the arrival of a

neuronal action potential at the neuromuscular junction.


24/91


25/91

11

muscle tissue. This is a comparatively accurate and localized method of EMG measurement.

For HCI applications this is not practically feasible. Second method of EMG measurement is

sEMG or surface EMG measurement. Here a surface electrode is used for acquiring the EMG

signals. The basic principle used in sEMG is that the electrical activity of muscles surfaces

out through the skin.

3.5 FACTORS AFFECTING EMG

Figure 3.3 Schematic flow diagram showing the factors affecting EMG signal

Extrinsic factors are caused due to the sensor design or the placement of the sensor. Fig. 3.3

shows all the intrinsic and extrinsic factors that affect the myo-signal and EMG respectively.

The figure also shows the interconnections between the factors. The deterministic factors are

the ones that can be identified and can be corrected. The EMG amplitude and frequency are

the two variables which changes according to these causative factors. The force, muscle

activation muscle fatigue and other physical details which need to be inferred from the EMG

signal is done by the analysis of the two variables namely amplitude and frequency of the

EMG signal.

It is to be noted that there are several intermediate factors such as signal cross talk,

conduction velocity and deterministic factors such as MUAP amplitude and frequency,

number of active MU which are caused by the causative factors, account for the variation in

EMG signal parameters.


26/91

12

CHAPTER 4

SIGNAL ACQUISITION AND CONDITIONING


27/91

13

The myo-signals generated by the body have to be acquired into the system for processing

and classification. Thus it becomes a typical data acquisition task which requires acquisition

of signals, conditioning of acquired signal and sending the signals to the computer in digital

form.

4.1 DATA ACQUISITION

Data acquisition (DAQ) is the process of sampling of real world physical conditions and

conversion of the resulting samples into digital numeric values that can be manipulated by a

computer. A data acquisition and data acquisition system typically involves the conversion of

analog waveforms into digital values for processing [22,23].

Block diagram of a typical Data Acquisition System is shown below.

The components of a data acquisition systems include:

Sensors that convert physical parameters to electrical signals.

Signal conditioning circuitry to condition and to convert sensor signals into a form

that can be converted to digital values. Acquisition Hardware consisting of Analog-to-digital converters, which convert the

conditioned sensor signals to digital values.

Computer, containing specific software, to comprehend the incoming digital data and

to enable further processing.

The signal conditioning block is very critical in a data acquisition system. It must be capable

of amplifying the signal in order to increase the power of the signal and filter out the noise

present in the signal [38]. The signals obtained from the body are of very low amplitude andpower. They contain noise which can degrade the quality of the signal. The Analog-to-Digital

Figure 4.1 Data Acquisition System Block Diagram


28/91

14

Converter (ADC) is generally equipped with sample-and-hold circuitry and converts the

analog samples into digital values. The digital data is then sent to the computer through

various ports such as parallel port, serial port, USB or through sophisticated, high speed

interfaces such as PCI, GPIB etc.

Generally, all computers are equipped with sound cards which enable connectivity to

microphones and speakers. The sound cards contain high speed, and high resolution Digital-

to-Analog Converters (DACs) and ADCs which can also be used as an interface to send

analog signals into the computer. In our project, we make use of the sound card that is present

in the computer to send the myo-signals after conditioning.

4.2 ELECTRODES AND ITS CLASSIFICATION

An electrode is an electrical conductor used to make contact with a non metallic part of a

circuit. Electrodes can be broadly classified into two categories namely,

1. Invasive type

2. Non Invasive type

4.2.1 INVASIVE ELECTRODES

Invasive electrodes are inserted into the muscle directly piercing the skin. The EMG signal

acquired is highly localized and devoid of most of the noises attributed to the EMG signals.

There are two types of invasive electrodes that are commercially available. They are Fine

wire type and Needle type [17,18].

For HCI applications, invasive techniques are not practically feasible.

4.2.2 NON INVASIVE ELECTRODES

Non invasive electrodes are mainly in the form of surface electrodes. These electrodes are

placed on the surface of the skin and the signals are taken from there. These electrodes

require a conducting gel to be applied on the surface of the skin so as to assist in the

conduction of the signal from the muscles to the electrode. Generally they are pre-gelled

silver / silver chloride electrodes and are disposable and are cheaper than other electrodes.


29/91

15

Figure 4.2 Surface Electrodes

The electrodes are made of plastic foam material and have a silver plated disk with a snap to

attach the leads. As mentioned above it has a gel which is generally an electrolyte like the

silver, silver chloride electrolyte. It also has an adhesive surface to attach it to the skin. There

are quite a few factors that affect the quality of the signal that is obtained out of the electrode.

The location of the electrode plays a very important role in determining the quality of the

signal achieved. The electrode skin interface contributes to noise if it is not kept clean and

dust free. The signals from the muscles in close proximity to the muscle in consideration also

contribute to the error in the signal. This concept of other muscle signals interfering is called

Cross talk. The main reason as to why this electrode is preferred over the invasive type of

electrode is that it is not necessary to make incisions in order to place the electrodes. Also it

is cheaper than any of the invasive electrodes taken into consideration. The disadvantage of

this electrode over the invasive electrodes is that these electrodes have very little resistance to

noise and the contribution of error due to cross talk is very high.

4.3 SURFACE EMG (sEMG)

The small electrical signal which comes from the active muscles is detected by electrodes

placed on the skin directly above the muscles. The procedure that measures the muscle

activity from the skin is referred to as sEMG. There are many locations in the body where the

electrical activity of the muscles surfaces out.

To control the manipulator, proper forearm muscles have to be selected. Figure 4.3 shows the

various electrode placement sites in the human body. Some of the muscles involved in the

forearm movements are: Wrist extensor group, wrist flexor group, Flexor Carpi Radialis,

Flexor Carpi Ulnaris, Brachioradialis, etc.


30/91

16

Figure 4.3 Electrode placement positions in the body

4.4 GESTURE SELECTION AND PLACEMENT OF ELECTRODES

Various movements of the wrist were considered as the gestures. Five movements of the

wrist were identified. These are:

1. Flexion of wrist at an angle of one third of maximum possible movement

2. Flexion of wrist at an angle of two third of maximum possible movement

3. Flexion of wrist at maximum possible angle.

4. Extension of the wrist backwards at a maximum possible angle.

5. Cylindrical grabbing of an object.

Wrist movement was selected because of the relative simplicity to restrict the degree of

freedom to just one and also because proper classification of these gestures could lead to

many practical uses. Muscles involved in these movements are the flexor carpi radialis and

flexor carpi ulnaris.

Figure 4.4 Muscles in the arm


31/91

17

These two muscles can be identified from outside the skin surface by performing the wrist

flexion and extension and identifying the group of muscles that move.

4.5 SURFACE EMG SENSOR CHARACTERISTICS

In addition to the electrical characteristics of the sensor, the design of the sensor should

address other practical factors such as:

1. Effectiveness of the electrical contact between the electrode and the skin

2. Facility of attaching the sensor to the skin

3. Durability of the adhesion to the skin

4. Insensitivity of the electrical and mechanical performance to the presence of sweat.

5. Insensitivity to movement artifact

6. Ease of use on small muscles

4.6 ELECTRODE SELECTION

Figure 4.5 Surface electrodes used and delsys electrodes

Two types of sEMG electrodes were under our consideration for this project. These are the

Delsys differential electrodes and the common monitoring surface electrodes. In the

preliminary stage itself it was decided to select the common surface electrodes over the

Delsys electrodes because of the following reasons:

1. Delsys electrodes are highly sensitive to the location of the electrode on the muscle.

Since positioning the electrodes with a high level of accuracy is not practically feasible, the

use of Delsys electrodes is a disadvantage

2. The high sensitivity provided by the Delsys electrodes is not required for a gesture

recognition system. So the use of these electrodes are not cost efficient.


32/91

18

The common surface electrodes chosen were then used in differential configuration to

eliminate common mode noise. A reference electrode was kept on the forehead where there is

very little muscle activity.

The electrodes are circular is shape as given in figure. In a differential configuration the

distance between the electrodes is more than two centimetre normally. But optimally we need

to obtain one centimetre distance between the electrodes. As shown in figure, the electrodes

are cut on one side, i.e. the plastic part of the electrodes with the adhesive, so that when they

are placed side by side the distance obtained between them is one centimetre.

Figure 4.6 Surface electrodes with proper spacing

4.7 NOISE ASSOCIATED WITH SURFACE EMG SIGNALS

The EMG signals are acquired using the surface electrodes. There may be many noise related

to the placement of electrodes and movement of muscles. Electromagnetic radiation, inherentnoise in electronics, and motion artifact will adequately cover the types of noise that will be

encountered in obtaining the EMG signal. For maximizing the signal output signal-to-noise

ratio must be high, this can be obtained by minimizing the above factors.

4.7.1 ELECTROMAGNETIC RADIATION

The magnitude of the noise at the skin surface can be in the range of 3 times that of the EMG

signal making its removal imperative. Any electromagnetic device generates and maycontribute noise. The surfaces of the bodies are constantly inundated with electric-magnetic

radiation and it is virtually impossible to avoid exposure to it on the surface of the earth. The

dominant concern for the ambient noise arises from the 60 Hz (or 50 Hz) radiation from

power sources. A high Q value notch filter would suffice for removing this interference,

however; because the desired signal has energy located at this frequency a differential

amplification process is a superior solution to this problem. The design that was implemented

for this project uses just such a method. Figure below shows how a differential amplifier can


33/91

19

be used to remove noise. Due to the fact that the electromagnetic interference is common to

both electrodes subtracting the signals can remove the commonalities in the signal.

4.7.2 ELECTRONICS IMPERFECTION

There are some inherent noise in the electronics components used for the detection and

recording of EMG signals. All electronics equipment generates electrical noise. This noise

has frequency components that range from 0 Hz to several thousand Hz. This noise cannot be

eliminated; it can only be reduced by using high quality electronic components, intelligent

circuit design and construction techniques. The basic circuit includes AD620 and UAF42.

These produces very low amplitude noise which cannot be eliminated during acquisition but

the noise is removed by thresholding the signal acquired.

4.7.3 BASELINE NOISE

Noise due to the skin- electrode interface is reduced by effective skin preparation. The

baseline noise originates in the electronics of the amplification system and at the skin-

electrode interface. The ionic exchange between the metal in the electrode and the

electrolytes in the salts of the skin generates an electro-chemical noise. The magnitude of

noise is proportional to the square root of the resistance of the electrode surface. Thus, it can

be reduced by increasing the electrode area and by cleaning the electrode surface, but it

cannot be eliminated.

Figure 4.7 Baseline noise and EMG signal

The thermal-noise is generated by the first stage of the amplifiers and is due to a physical

property of the semiconductors. It also cannot be eliminated. Both noises are referred to as 1/fnoises, with the amplitude of the frequency spectrum greatest at 0 Hz and continuously


34/91

20

decreasing with increasing frequencies. The electrochemical noise is generally greater than

the thermal noise.

4.7.4 MOTION ARTIFACT

The two main sources of motion artifact are the interface between the detection surface of the

electrode and the skin and the movement of the cable connecting the electrode to the

amplifier. This motion usually causes lower frequency noise in the range of 0 20 Hz. The

removal of this motion artifact is accomplished via a simple high pass filter with a cut-off

frequency somewhere between 15-40 hertz. The removal of this noise would lead to loss of

signal so there must be a trade off so as to optimize between the signal and the noise.

4.7.5 INHERENT INSTABILITY OF THE SIGNAL

The amplitude of the EMG signal is quasi-random in nature. The frequency components

between 0 and 20 Hz are particularly unstable because they are affected by the quasi-random

nature of the firing rate of the motor units which, in most conditions, fire in this frequency

region. Because of the unstable nature of these components of the signal, it is advisable to

consider them as unwanted noise and remove them from the signal. The removal of first 5000

samples is done to prevent the signal being affected by noise. The first 5000 samples are

initialised to zero and the remaining samples from 5000 to 8000 are considered as the signal.

There is also an adaptive way to find the noise according to the amplitude level.

4.7.6 PHYSIOLOGICAL NOISE

Noise from the other process occurring in the body also affect the EMG signals (i.e.) ECG,

EOG and respiratory signals also contribute to the noise. This noise cannot be eliminated but

judicious placement of electrodes can reduce it.

4.7.7 CROSS-TALK

The use of large electrode area and large inter-electrode spacing invariably leads to detection

of cross-talk which is often misinterpreted as activity from the monitored muscle. In clinical

applications this misunderstanding may lead to false diagnosis. In the research field it may

lead to a basic misunderstanding of the performance of the monitored muscle. By using

smaller electrodes and shorter inter-electrode spacing, the cross-talk signal would be

substantially smaller.


35/91

21

4.7.8 POWER LINE NOISE

All the circuits need power supply for operation. The power supply used in India is 230V,

50Hz. So there is a 50Hz component in the signal which acts as a major contributor for the

noise. The use of batteries considerably reduces this noise level.

4.7.9 NOISE IN ELECTRODE CONNECTING WIRES

The leads or the connecting wires from the electrodes carry the signals from the body surface

to the signal acquisition board. The raw signal is most susceptible to noise during this

transmission. Normally coaxial cables are used to reduce noise. Coaxial cable is an electrical

cable with an inner conductor surrounded by a flexible, tubular insulating layer, surrounded

by a tubular conducting shield. It has two conductors, the central wire and the tubular shield.

At any moment the current is travelling outward from the source in one of the conductors,

and returning in the other.

Figure 4.8 Coaxial cable

4.8 ELECTRODE LOCATION AND AMPLITUDE VARIATION

The location of the sensor on the muscle renders a dramatically different surface EMG signal

characteristics. Note that locating the sensor in the proximity of the tendon origin, the

innervation zone, and the perimeters of the muscle yields lower amplitude signals.

Figure 4.9 Electrode Location and Signal Amplitude Variation


36/91

22

The fibres in the middle of the muscle have a greater diameter than those at the edges of the

muscle or near the origin of the tendons. Because the amplitude of action potential from the

muscle fibres is proportional to the diameter of the fibre, the amplitude of the EMG signal

will be greater in the middle of the muscle. A sensor located on the innervations zone will

detect the cancellation of the action potentials travelling in opposite direction, and will

generally have lower amplitude. The preferred location is away from all these boundaries,

towards the middle of the muscle surface.

4.9 IDEAL LOCATION OF THE ELECTRODES FOR A HIGH

FIDELITY SIGNAL

Figure 4.10 indicates the preferred location for placing the sensor in the middle of the musclesurface and as far away as possible from the innervation zones and tendon origins. The small

yellow striped areas indicate the innervation zones which in large muscles are located around

the periphery.

Figure 4.10 Ideal locations of electrodes

4.10 IMPORTANCE OF ELECTRODE SPACING

By maintaining a fixed inter-electrode spacing, the bandwidth of the surface EMG signal will

remain constant. The band-width of the sensor determines how much of the signal energy and

the noise energy the acquired signal constitutes. If the electrode spacing is varied as may

occur with sensors that have separate electrodes that may be attached with variable spacing at

each application, then the information content in the acquired surface EMG signal will not be

constant and comparison among muscles and subjects will be unreliable. A small inter-

electrode spacing is preferable as it will reduce the amount of crosstalk signal detected from

adjacent active muscles. The greater the spacing of electrodes greater is the difference of the

propagating cross-talk signal. Generally one centimetre is the preferred compromise.


37/91


38/91

24

4.12.2 FILTERS

The maximum power of the EMG signal is contained in the frequency range of 10 to 300 Hz

[4]. Since the range is small, a combination of analog filer and digital filter is used in the

signal acquisition process. An analog low pass filter with a cut-off frequency of 1000 Hz is

chosen to filter the incoming EMG signals from the body. This is to ensure a safe margin so

that the important information in the signal is not lost. Further filtering is done in the software

by employing digital filtering techniques (Refer chapter 5).

Active filters are generally employed for low frequency applications to reduce size and cost.

Active filters are designed using OP-Amps, Resistors and Capacitors. Manual soldering of

these filter components on a PCB may add up the noise in the circuit and moreover future

modifications may become impossible. Hence a single chip filter such as the Universal

Active Filter UAF42 can be used. It can be configured for various types of high-pass, low-

pass, and band-pass filters [Refer appendix B.2]. The filters implemented with the UAF42 are

time-continuous, free from the switching noise and aliasing problems of switched-capacitor

filters. The implementation of filter is very easy requiring only few external resistors to

implement the various filter configurations. Each UAF42 IC can be used to realize a filter of

order 2.

Figure 4.12 Internal Structure of Universal Active Filter UAF42

Figure 4.13 UAF42 Pin Diagram


39/91

25

A DOS-compatible filter design program called FILTER42allows easy implementation of

many filter types, such as Butterworth, Bessel, and Chebyshev. A fourth, uncommitted FET-

input op amp (identical to the other three) can be used to form additional stages, or for special

filters such as band-reject and Inverse Chebyshev.

4.12.3 FILTER42 PROGRAM

The FILTER42 program is a DOS-compatible program that guides through the filter design

process and automatically calculates the component values. Using this program, Low-pass,

high-pass, band-pass, and band-reject (or notch) filters can be designed. FILTER42 supports

the three most commonly used all-pole filter types: Butterworth, Chebyshev, and Bessel.

4.12.4 FILTER42 PROGRAM STEPS

The program gets the input such as filter type, order, cut-off frequency etc from

the user.

Figure 4.14 Design guide

The program then shows a frequency plot of the designed filter and then displays

the list of component values.

Figure 4.15 Filter component values generated by FILTER42


40/91


41/91


42/91

28

4.14 SOUND CARD INTERFACE

A sound cardis a computer expansion card that can input and output sound under control of

computer programs. Many computers have sound capabilities built in, while others require

these expansion cards if audio capability is desired. Most sound cards have a line-in

connector where the sound signal from a cassette tape recorder or similar sound source can be

input. Another typical external connector is the microphoneconnector, for connecting to a

microphone or other input device that generates a relatively lower voltage than the line in

connector. [2]

A typical sound card has two hardware channels. These two hardware channels are used to

input the amplified and filtered EMG signals into the computer. The Simulink file in

MATLAB initializes the sound card with the required minimum sampling frequency (8000

samples/second) and initiates acquisition by creating objects.

Figure 4.22 Block Diagram of Sound Card Interface

Each computer has different sound card specifications. Hence a preliminary study is required

to study about the sound card used and to design the data acquisition system accordingly. In

many cases, impedance matching has to be done to ensure the integrity of the signals. In this

project, the board is designed to meet up with the specifications of sound card used.

The outputs from the data acquisition system boards are sent to the line-in jack of the sound

card through a standard 3.5 mm stereo microphone jack. The 3.5 mm stereo microphone jack

consists of three terminals. Two terminals are connected to the outputs from 2 data

acquisition boards and the third terminal is the ground. The connection from the board to the

jack is made using a 2 core shielded cable.


43/91

29

CHAPTER 5

SIGNAL PRE-PROCESSING


44/91

30

The signals obtained from the sound card have to be filtered further by employing digital

filters in MATLAB. By employing digital filters, the filter specifications can be modified

anytime depending on the application.

5.1 DIGITAL FILTERS

Two digital filters are used for filtering of signals. 95% of the energy of EMG signal lies in

the frequency range of 10 to 300 Hz. Hence in the Simulink in MATLAB there are two

digital filters connected in series and connected to scope to view the output of the performed

action.

Figure 5.1 Digital filter block diagram

High pass filter specifications: IIR, Butterworth Filter, and Cut off 10 Hz

Low pass filter specifications: IIR, Butterworth Filter, and Cut off 300 Hz

Figure 5.2 Simulink system for data acquisition


45/91

31

Since there are two channels one in the upper muscle in the arm and other in the lower part of

the arm. These two channels are to get the different movements. Hence we get two scope

values as the output of the movement. The RMS values of both the output are calculated.

These values are taken as the feature value. The two filters connected in series are a high pass

filter and a low pass filter. The high pass filter used here passes all the value of frequencies

above 10 Hz since the values of the frequencies below 10 Hz has many unwanted noise

signals which does not give the actual signal. This filter is connected in series with a low pass

filter which passes all the frequency value below 300 Hz. Hence these two filters together

form a band pass filter of the band pass frequency of 10-300 Hz.

5.2 ADAPTIVE THRESHOLDING

Since the EMG signal does not have any regular pattern as ECG there is a need for adaptive

thresholding for removal of noise. EMG signals are random signals so the noise is also

random. Hence in the part where gesture is obtained also contains some random noise. To

obtain the actual value of the amplitude of the gesture, the noise from that part of the signal

must be removed.

5.3 CROPPING OF THE FILTERED SIGNAL

The exact gesture is required to identify the gesture without any other noise. As mentioned

above there are many types of noise which affect the quality of the signal, so only the gesture

part must be extracted. The sound card has a sampling rate of 8000 but the first 5000 samples

are initialized to 0 so as to remove the initial noise. Using thresholding method the baseline

noise is also removed. The part after the gesture is removed by comparing the amplitude

values of the previous values. By comparing the value with the database values the gesture is

identified.


46/91

32

CHAPTER 6

GESTURES AND SIGNALS


47/91


48/91

34

The signals obtained for the various gestures are shown below.

[a]

[b]

[c]

[d]


49/91

35

[e]

[f]

Figure 6.3 EMG signals obtained for [a] No movement, [b] Up1 movement, [c] Up2 movement,

[d] Up3 movement, [e] Down movement, [f] Grab movement

Figures [e] and [f] show that the channel two is active only during the down and grab

gestures. This shows that the muscle corresponding to the channel two is active only during

these two movements. Figures [b],[c] and [d] shows that the amplitude of the signals

increases as the angle of the wrist increases. The signal is almost zero when no move

movement is made.


50/91

36

CHAPTER 7

CLASSIFICATION


51/91

37

Classification is the assignment of a class label to an input object. There are basically two

types of classification: Binary classification and Multiclass classification. Multiclass

classification is used in this project since there are more than two classes or gestures.

Classification is done based on the features of the input object. Features are the individual,

measurable heuristic properties of the phenomena being observed. Choosing discriminating

and independent features is a key to any pattern recognition algorithm being successful in

classification.

7.1 ROOT MEAN SQUARE (RMS) VALUE

The features that we are considering are of two types. They are spectral and statistical. The

spectral feature space consists of the Fourier coefficients, the spectral intensity and the

Fourier variance of the input signal. The statistical feature space consists of the following

features: Maximum, Minimum, Mean value, Variance, Signal length, Root Mean Square.

The feature used in this project is the RMS value [54]. This is because the difference in the

RMS value is much more significant for different actions performed for the given hardware

setup. RMS value is the square root of the arithmetic mean of the squares of the original

values. The quality of the spectral features isnt satisfactory for this hardware and hence just

one feature is used to classify our signals [28].

Figure 7.1 Scatter Plot for the various movements taken into consideration.

However, when two channels are used, the separation between the two RMS values is more

evident and it is possible to separate each class from the other based on this.


52/91

38

7.2 SUPPORT VECTOR MACHINES

Support Vector Machines is a tool that is frequently used in order to classify a particular input

data into various groups. In order to ensure proper classification, the classifier is trained with

some training data [27, 32]. The groups for classification are:

1. Down Movement 4. Up 1 Movement

2. Up 2 Movement 5. Up 3 Movement

3. Grab Movement 6. No Movement

The given input data can be classified using two approaches which are:

1.

One vs. Rest approach

2. One vs. One approach

In this project we are using the One vs. One approach because it is a much less time

consuming and easier approach to achieve the required goal. The feature we are using in this

project in order to assist in the classification is the Root Mean Square value of the signal.

This RMS value provides visible difference in values for the various actions performed.

Initially, a database is made using the RMS values of the various signals acquired for their

respective action. This database is then used to train the SVM classifier. There are 2 channels

in the acquisition board we are using and hence we get 2 coordinates i.e. RMS values for all

the signals. The plot of the various RMS values is shown in Fig 7.1.

Using the one vs. one approach, we have optimized the number of classifiers required to 6.

The following are the classifiers we use:

1. Down vs. Up3 4. Up1 vs. Up2

2. Up1 vs. Up3 5. Up2 vs. Up3

3. Up3 vs. Grab 6. Down vs. Grab

These classifiers were all decided based on the experiments performed. The reasons can be

observed in Fig.6.1 itself. The muscles used for the down movement are different from those

used in the up movement. This gives us sufficient information to separate the down from the

up3 movement. Then, we need to separate the up1 movement from up2. If up1 movement is

the output then we leave it as it is. If up2 movement is the output, we further verify it with

up3 movement to ensure that it is up2 movement. If it so happens that the output is up3

movement, it is further verified to ensure that it isnt grab movement. This way at each step

there is elimination of certain movement. Fig 7.2 describes the exact algorithm.


53/91

39

Figure 7.2 SVM algorithm for efficient classification.

Using the above algorithm chart, it is possible to classify the signals into their respective

groups.

Figure 7.3 SVM Classifier for Up2 & Up3 Movement Figure 7.4 SVM Classifier for Down & Grab Movement

Figure 7.5 SVM Classifier for Down & Up3 Movement Figure 7.6 SVM Classifier for Up3 & Grab Movement


54/91


55/91

41

The RMS value of the input channels are determined and taken as the coordinates of the

particular input signal. The Euclidean Distance formula is used to determine the distance of

the input data point from the clusters up1, up2, up3, down and grab. Based on the distance

acquired, the minimum value of the distances is calculated and is used to classify the data

into the group.

Figure 7.9 Diagram depicting KNN for classification

Figure 7.10 The algorithm for classification using KNN Classifier

Using the above algorithm it is possible to classify the input data using the K Nearest

Neighbourhood Classifier.


56/91

42

7.3.1 RESULT FOR KNN

Table 7.2 Confusion Matrix for KNN Classifier

KNN classifier efficiently classifies the signal since it finds the distance between the new

sample and the cluster values. The accuracy is almost similar to SVM.

7.4 ARTIFICIAL NEURALNETWORK

Artificial neural networks are non linear information processing devices, which are built from

interconnected elementary processing devices called neurons. An artificial Neural Network

(ANN) is an information processing paradigm that is inspired by the way biological nervous

systems, such as the brain, process information. The novel structure of the information

processing system is composed of a large number of highly interconnected processing

elements working in union to solve specific problems. An ANN is configured for a particular

application through a learning process.

7.4.1 RADIAL BASIS FUNCTION

The RBF networks are single-layer perceptron networks that are commonly used to perform

classification tasks. The performances of a RBF classifier will depend on the success of

tuning the weights and the centres using the training set. A RBF classifier performs a

clustering operation on the symptom vectors in the training set. If the symptom vectors

corresponding to a pre-specified state naturally group themselves in more than one cluster in

the m-dimensional space, then RBF classifiers need as many centres per category as the

number of natural clusters associated with that category to solve the problem. In radial basis

function the goal and the spread constant is specified, which gives a more accurate output by

classifying the input correctly [7].


57/91


58/91

44

7.4.3 ALGORITHM OF RADIAL BASIS FUNCTION

Figure 7.13 Algorithm of RBF used in MATLAB

7.4.4 RESULT FOR ANN

Table 7.3 Confusion Matrix for ANN Classifier

ANN classification method is less efficient compared to KNN and SVM since it depends

on the number of neuron centres. The accuracy is lower than the other classification

techniques still it provides an accuracy level of 92%.


59/91

45

CHAPTER 8

PERFORMANCE ANALYSIS


60/91

46

The real time efficiency of the classifiers and the accuracy of classification will be the scope

of this chapter.The real time efficiency refers to the delay in the response of the classifier to a

particular action performed. In this case, the real time efficiency of the K Nearest

neighbourhood classifier is the most optimal. It takes approximately 1.2 milliseconds for the

response to be generated. Support vector Machines is also an effective means for

classification with a delay of about 4 milliseconds. In comparison, the Artificial Neural

Network takes 12 milliseconds to efficiently classify the input signal. The table given below

shows the details of the time required for classification for the three classifiers.

Table 8.1 Time required for classification

The table given below summarises the classification accuracy of the three classifiers.

Table 8.2 Classification accuracy of three classifiers

CLASSIFIER

CLASSIFICATION ACCURACY

Up1 Up2 Up3 Down Grab

SVM 94% 94% 98% 96% 96%

ANN 92% 92% 92% 90% 92%

KNN 96% 96% 96% 96% 96%


61/91

47

CHAPTER 9

MANIPULATOR DESIGN AND CONTROL


62/91

48

In this project, a simple Articulated Robot Manipulator is built to demonstrate the controlling

capability of the designed system. Articulated robots are powered by variety of means such as

DC motors, Stepper motors, Servo motors etc.

DC motors and stepper motors do not have a positional feedback systems incorporated in

them; hence they suffer from imbalance when the shaft moves due to excess load. Whereas a

servo motor has a potentiometer based positional feedback to correct the error generated due

to excessive loading conditions.

9.1 THE SERVO MECHANISM

A servomechanism or servo is an automatic device that uses error-sensing feedback to correct

the performance of a mechanism. The term correctly applies only to systems where the

feedback or error-correction signals help control mechanical position or other parameters.

Figure 9.1 The Servomechanism in an RC Servo Motor

A common type of servo provides position control. Servos are commonly electrical or

partially electronic in nature, using an electric motor as the primary means of creating

mechanical force [8]. Usually, servos operate on the principle of negative feedback,where

the control input is compared to the actual position of the mechanical system as measured by

some sort of transducer at the output. Any difference between the actual and wanted values

(an "error signal") is amplified and used to drive the system in the direction necessary to

reduce or eliminate the error. This procedure is one widely used application ofcontrol theory.
http://en.wikipedia.org/wiki/Electric_motorhttp://en.wikipedia.org/wiki/Forcehttp://en.wikipedia.org/wiki/Negative_feedbackhttp://en.wikipedia.org/wiki/Transducerhttp://en.wikipedia.org/wiki/Control_theoryhttp://en.wikipedia.org/wiki/Control_theoryhttp://en.wikipedia.org/wiki/Transducerhttp://en.wikipedia.org/wiki/Negative_feedbackhttp://en.wikipedia.org/wiki/Forcehttp://en.wikipedia.org/wiki/Electric_motor


63/91

49

9.1.1 RC SERVOS

Figure 9.2 A typical RC Servo

RC servos are hobbyist remote control devices typically employed in radio-controlled

models,where they are used to provideactuation for various mechanical systems such as the

steering of a car, the flaps on a plane, or the rudder of a boat.

RC servos are composed of an electric motor mechanically linked to a potentiometer. Pulse-

width modulation (PWM) signals sent to the servo are translated into position commands by

electronics inside the servo. When the servo is commanded to rotate, the motor is powered

until the potentiometer reaches the value corresponding to the commanded position. The RC

servos have very high torque due to heavy gear reductions inside them.

Due to their affordability, reliability, and simplicity of control by microprocessors, RC servos

are often used in small-scalerobotics applications. (Refer appendix B.3)

9.1.2 CONTROLLING A SERVO MOTOR

Figure 9.3 Timing characteristics of a servo control PWM signal

The servo is controlled by three wires: ground, power and control. The servo will move based

on the pulses sent over the control wire, which set the angle of the actuator arm. The servo

expects a pulse every 20 ms in order to gain correct information about the angle. The width of

the servo pulse dictates the range of the servo's angular motion.
http://en.wikipedia.org/wiki/Radio-controlled_modelhttp://en.wikipedia.org/wiki/Radio-controlled_modelhttp://en.wikipedia.org/wiki/Actuationhttp://en.wikipedia.org/wiki/Pulse-width_modulationhttp://en.wikipedia.org/wiki/Pulse-width_modulationhttp://en.wikipedia.org/wiki/Roboticshttp://en.wikipedia.org/wiki/Roboticshttp://en.wikipedia.org/wiki/Pulse-width_modulationhttp://en.wikipedia.org/wiki/Pulse-width_modulationhttp://en.wikipedia.org/wiki/Actuationhttp://en.wikipedia.org/wiki/Radio-controlled_modelhttp://en.wikipedia.org/wiki/Radio-controlled_model


64/91

50

A servo pulse of 1.5 ms width will set the servo to its "neutral" position, or 90. For example

a servo pulse of 1.25 ms could set the servo to 0 and a pulse of 1.75 ms could set the servo

to 180. The physical limits and timings of the servo hardware varies between brands and

models, but a general servo's angular motion will travel somewhere in the range of 180 -

210 and the neutral position is almost always at 1.5ms.

9.2 MANIPULATOR DESIGN AND CONSTRUCTION

The manipulator is designed in such a way that it resembles a human wrist and also to

perform a simple pick-and-place operation. The manipulator has two joints. It is constructed

using plywood and the links are driven by 2 servo motors. One servomotor to move the wrist

left and right and another servo motor to make the gripper, open and close.

The CAD drawings of the manipulator are shown below.

Figure 9.4 Manipulator Base - Front and Side View

Figure 9.5 Manipulator gripper - front and side view


65/91

51

This manipulator can demonstrate a small pick-and-place operation.

Figure 9.6 Manipulator-complete view

9.3 MANIPULATOR CONTROL

The servo motors in the manipulator require Pulse Width Modulated (PWM) signals to

accurately position the motor shaft. This is achieved by using a PIC16F877A microcontroller.

The PIC16F877A controller falls under the mid-range series in the PIC family of

microcontrollers. These controllers are manufactured and distributed by Microchip

Technology Inc, USA (Refer appendix B.4).

9.3.1 PIC16F877A PERIPHERALS USED

Two Capture, Compare, PWM modules - PWM maximum resolution is 10-bit

Universal Synchronous Asynchronous Receiver Transmitter (USART/SCI)

The PIC16F877A controller has two independent hardware PWM generators which can be

independently initialized and be used to control the two servo motors in the manipulator.


66/91


67/91

53

The schematic of the PIC16F877A controller, connected to control the servo motors, is

shown below.

Figure 9.8 Circuit Diagram of PIC16F877A based Manipulator Controller

The schematics and layout is designed using EAGLE 5.7.0 software.

Figure 9.9 LayoutTop Figure 9.10 LayoutBottom


68/91

54

The PIC is clocked using a 4 MHz crystal oscillator. Since laptop computers do not come

with a serial port connector, a USB-to-Serial converter is used to communicate with the PIC

microcontroller. To convert the RS-232 signals into TTL signals, a MAX232 based level

shifter is employed.

Figure 9.11 A Typical USB-to-Serial Converter

The MAX232 converts the RS232 +/- 12 volt signals to proper TTL 0 to 5 volt signals. The

MAX232 chip is manufactured by Maxim Semiconductors.

Figure 9.12 MAX232 Pin Diagram Figure 9.13 MAX232 circuit implementation

Fig 9.13 shows the MAX232 circuit implementation. The capacitors in the circuit acts as

charge pumps to step-up the voltage form 5 volts to 12 volts.

9.4.2 PIC16F877A- SERVO CONTROL ALGORITHM

(The code for the servo control is attached in the appendix)

Step 1: Initialize the USART module for 9600Bd

Step2: Initialize the PWM module for 250Hz

Step3: Start both the PWM modulesStep4: If data from the UART buffer is A, set the wrist servo to position UP1


69/91

55

Else goto step5

Step5: If data from the UART buffer is B, set the wrist servo to position UP2

Else goto step6

Step6: If data from the UART buffer is C, set the wrist servo to position UP3

Else goto step7

Step7: If data from the UART buffer is D, set the wrist servo to position DOWN

Else goto step8

Step8: If data from the UART buffer is O, set the Gripper servo to position OPEN

Else goto step9

Step9: If data from the UART buffer is G, set the Gripper servo to position GRAB

Else goto step10

Step10: If data from the UART buffer is N, set both servos to NO MOVEMENT

Else goto step4

Step11: END

The system is designed in such a way that a single GRAB movement can perform GRAB

movement as well as OPEN movement in the manipulator. When a single GRAB movement

is made, the system identifies it and the manipulator performs a GRAB operation. When a

GRAB gesture is done again, the system identifies that the manipulator is already grabbing

something and hence it performs an OPEN/DROP operation.

9.5 MANIPULATOR CONTROL USING HAND GESTURES - RESULTS

[a]


70/91

56

[b]

[c]

[d]


71/91

57

[e]

[f]

[g]

Figure 9.14 Gesture performed by the user and the corresponding manipulator movement for

[a] No movement, [b] Up1 movement, [c] Up2 movement, [d] Up3 movement, [e] Down movement,

[f] Grab movement


72/91

58

CHAPTER 10

CONCLUSION


73/91

59

A small, cost effective and size optimized gesture recognition system, which can be easily

worn and carried by the user, has been built. The system is capable of acquiring 2 channels of

EMG signals from the body, with minimum noise levels, pre-process them and classify the

hand gestures made.

The use of non-invasive electrodes has proved to be a very convenient technique for portable

Human Computer Interface applications. These cheap non-invasive electrodes can be used

two or three times and then replaced with a new one.

Three classifiers have been implemented and out of the results of the three classifiers, best

two is taken as the final class. This form of classifier fusion results in an increased accuracy.

CLASSIFIERCLASSIFICATION ACCURACY

Up1 Up2 Up3 Down Grab

SVM 94% 94% 98% 96% 96%

ANN 92% 92% 92% 90% 92%

KNN 96% 96% 96% 96% 96%

This table gives the results obtained after repeated measurements.

The capability of this gesture recognition system has been demonstrated by controlling an

articulated robotic arm with two DOF. The manipulator is directly controlled by the gestures

made by the user. This manipulator can be used in many application such as pick and place

operations, manipulation of objects in hazardous environment etc.


74/91

60

CHAPTER 11

FUTURE SCOPE


75/91

61

The use of bio-signals is of very high significance in human computer interaction or human

machine interaction. EMG itself has a wide range of possible applications. The possible

projects that can be extended from our project are given below:

Control of prosthetic arm Prosthetic arm can be controlled by classifying the EMG signals

generated by the muscles of an amputee.

Hand-written character recognitionHand-written characters can be recognized using EMG

signals obtained from the forearm of the writer.

Control of robotic manipulatorEMG signals can be used to control robotic manipulators for

medical and industrial applications instead of joystick.


76/91

62

REFERENCES

[1] C. J. De Luca, 1997, The use of surface electromyography in biomechanics, Journal of

Applied Biomechanics, 13 (2): 135-163

[2]

Mohamed Sami Abdul Staar, Mohamed Omar Abdul Aziz, Nada Mohamed Mahmoud

abdul Latif, Mai Mohamed Ali Hamza, Ahmed Mamdoh, "Brain Computer Interface",

Cairo University, 2006.

[3]

Sijiang Du, Marko Vuskovic, Temporal vs. Spectral Approach to Feature Extraction

from Prehensile EMG Signals, San Diego State University.

[4] Y.Yazama, Y.Mitsukura, M.Fukumi, N Akamatsu, Analysis and Recognition of Wrist

Motions by Using Multidimensional Directed Information and EMG signal, IEEE, 2004

[5]

T. Scott Saponas, Desney S. Tan, Dan Morris, Ravin Balakrishnan, Demonstrating the

Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces,

ACM, CHI2008.

[6] Jonghwa Kim, Stephan Mastnik, Elisabeth Andr, EMG-based Hand Gesture

Recognition for Real-time Bio signal Interfacing, ACM, IUT08.

[7] S .N.Sivanandam, S.Sumathi, S.N.Deepa, Introduction top Neural Networks Using

MATLAB 6.0, pp.10-29,212-218.

[8]

http://en.wikipedia.org/wiki/Servomechanism

[9] Andres Herrera, Andres Bernal, David Isaza and Malek Adjouadi, Design of an

Electrical Prosthetic Gripper using EMG and Linear Motion Approach, Center for

Advance Technology and Education Department of Electrical and Computer Engineering.

[10]Kevin R. Wheeler, Mindy H. Chang, and Kevin H. Knuth, Gesture Based Control and

EMG Decomposition. IEEE TRANSACTIONS ON SYSTEMS, MAN, AND

CYBERNETICS, VOL. 1, NO. 11, NOVEMBER 2005.

[11]

http://www.mathworks.com/access/helpdesk/help/techdoc/creating_guis/f2-998436.html[12]Marieb, Elaine; Katja Hoehn (2007). Human Anatomy & Physiology (7th Ed.). Pearson

Benjamin Cummings. p. 317.

[13]

Purves, Dale, George J. Augustine, David Fitzpatrick, William C. Hall, Anthony-Samuel

LaMantia, James O. McNamara, and Leonard E. White (2008). Neuroscience. 4th

ed.Sinauer Associates. pp. 7, 2728

[14]Newmark J (2007). "Nerve agents". Neurologist 13 (1): 2032

[15]Lawrence JH and De Luca CJ. The myoelectric signal versus force relationship in

different human muscles, Journal of Applied Physiology, 54: 1653-1659, 1983.


77/91


78/91

64

[32]Abdul Rahim Ahmad,Marzuki Khalid,Rubiyah Yusof, Machine Learning using Support

Vector Machines

[33]Kevin R. Wheeler, Mindy H. Chang, and Kevin H. Knuth, Gesture Based Control and

EMG Decomposition. IEEE TRANSACTIONS ON SYSTEMS, MAN, AND

CYBERNETICS, VOL. 1, NO. 11, NOVEMBER 2005.

[34]http://www.mathworks.com/access/helpdesk/help/techdoc/creating_guis/f2-998436.html

[35]

http://iweb.tntech.edu/scanfield/Canfield/me4140/06F/Robot%20Simulation

%20in%20Matlab%20V2.doc

[36]T. Scott Saponas, Desney S. Tan, Dan Morris, Ravin Balakrishnan, Demonstrating the

Feasibility of Using Forearm Electromyography for Muscle-Computer Interfaces, CHI

2008 Proceedings , Physiological Sensing for Input April 5-10, 2008 , Florence, Italy.

[37]

Jonghwa Kim, Stephan Mastnik, Elisabeth Andr, EMG-based Hand Gesture

Recognition for Real-time Biosignal Interfacing,Association for Computing Machinery,

978-1-59593-987-6/ 08/ 0001.

[38]

Gianluca De Luca, Fundamental Concepts in EMG Signal Acquisition, Rev.2.1, March

2003.

[39]Sebastian Bitzer, Patrick van der Smagt, Learning EMG control of a robotic hand:

Towards Active Prostheses, Proceedings of the 2006 IEEE International Conference on

Robotics and Automation, Orlando, Florida - May 2006.

[40]

Carlo J. De Luca, A Practicum on the Use of sEMG Signals in Movement Sciences,

Delsys Inc, ISBN: 978-0-9798644-0-7, 05/10/08.

[41]Kaveh Momen, Sridhar Krishnan, Real-Time Classification of Forearm

Electromyographic Signals Corresponding to User-Selected Intentional Movements

[42]For Multifunction Prosthesis Control, IEEE Transactions on Neural Systems and

Rehabilitation Engineering, Vol. 15, No. 4, December 2007.

[43]

Carlo J. De Luca, Surface Electromyography: Detection and recovery,

www.delsys.com.

[44]Y Su1, A Wolczowski, M .H. Fisher, G .D .Bell, D Burn, R Gao, Towards an EMG

Controlled Prosthetic Hand Using a 3D Electromagnetic Positioning System, IMTC

2005Instrumentation and Measurement Technology Conference Ottawa, Canada, 17-19

May 2005.

[45]Darrin. Young, Bradley D. Farnsworth, Ronald 1. Triol02, Wireless Implantable EMG

Sensor for Powered Prosthesis Control,IEEE-978-1-4244-2186-2/08, 2008


79/91

65

[46]M. B. I. Reaz, M. S. Hussain, F. Mohd-Yasin, Techniques of EMG signal analysis:

Detection, Processing, Classification and Applications, Biol. Proceed. Online 2006;8(1):

11-35. Doi: 10.1251/bpo115, March 23, 2006.

[47]Andrew W. Moore, K-means and Hierarchical clustering, School of Computer Science

[48]

Md. R. Ahsan, Muhammad I. Ibrahimy, Othman O. Khalifa, EMG Signal Classification

for Human Computer Interaction: A Review, European Journal of Scientific Research

ISSN 1450-216X Vol.33 No.3 (2009), pp.480-501.

[49]Laurene Fausett, Fundamentals of Neural Networks Architectures, Algorithms and

Applications, Pearson Education, 2009.

[50]

John.C.Platt, Sequential Minimal Optimization: A Fast Algorithm for Training Support

Vector Machines, Technical Report MSR-TR-98-14, April 21, 1998.

[51]

N. Cristianini and J. Shawe-Taylor, An Introduction to Support Vector Machines

Cambridge University Press, 2000.

[52]http://www.mathworks.com/access/helpdesk/help/toolbox/daq/f5-32960.html

[53]

http://en.wikipedia.org/wiki/Root_mean_square

[54]http://en.wikipedia.org/wiki/Classification_(machine_learning)

[55] http://en.wikipedia.org/wiki/Features_(pattern_recognition)
http://en.wikipedia.org/wiki/Root_mean_squarehttp://en.wikipedia.org/wiki/Features_(pattern_recognition)http://en.wikipedia.org/wiki/Features_(pattern_recognition)http://en.wikipedia.org/wiki/Root_mean_square


80/91

66

APPENDIX A

1. MATLAB CODE FOR GRAPHICAL USER INTERFACE

functionvarargout = GUI(varargin)% GUI M-file for GUI.fig

% Begin initialization code

gui_Singleton = 1;

gui_State = struct('gui_Name', mfilename, ...

'gui_Singleton', gui_Singleton, ...

'gui_OpeningFcn', @GUI_OpeningFcn, ...

'gui_OutputFcn', @GUI_OutputFcn, ...

'gui_LayoutFcn', [] , ...

'gui_Callback', []);

ifnargin && ischar(varargin{1})

gui_State.gui_Callback = str2func(varargin{1});

end

ifnargout

[varargout{1:nargout}] = gui_mainfcn(gui_State, varargin{:});

else

gui_mainfcn(gui_State, varargin{:});

end

% End initialization code

% --- Executes just before GUI is made visible.

functionGUI_OpeningFcn(hObject, eventdata, handles, varargin)

% Choose default command line output for GUI

handles.output = hObject;

% Update handles structure

guidata(hObject, handles);

% UIWAIT makes GUI wait for user response (see UIRESUME)

% uiwait(handles.figure1);% --- Outputs from this function are returned to the command line.

functionvarargout = GUI_OutputFcn(hObject, eventdata, handles)

% Get default command line output from handles structure

varargout{1} = handles.output;

% --- Executes on button press in start.

functionstart_Callback(hObject, eventdata, handles)

[y,k]=justclassifier();

ify==0

msgbox('NO MOVEMENT');


81/91


82/91

68

3. MATLAB CODE FOR READING THE DATA FROM THE

DATABASE

function [down,up3,up1,up2,grab]=justdaq()

up3=xlsread('D:\MATLAB\R2009a\work\Sound Card\Database\rmsdF.xls','UP3');

down=xlsread('D:\MATLAB\R2009a\work\Sound Card\Database\rmsdF.xls','DOWN');



grab=xlsread('D:\MATLAB\R2009a\work\Sound Card\Database\rmsdF.xls','GRAB');

4. MATLAB CODE FOR SVM TRAINING

function [dustruct,u12struct,uu1struct,u2ustruct,ugstruct,dgstruct] =

juststructure(down,up3,up1,up2,grab)

g=[-1*ones(1,length(down)) 1*ones(1,length(down))];

dustruct=svmtrain([down up3],g','showplot',true);

figure

u12struct=svmtrain([up1 up2],g','showplot',true);

figure

uu1struct=svmtrain([up3 up1],g','showplot',true);

figure

u2ustruct=svmtrain([up2 up3],g','showplot',true);

figureugstruct=svmtrain([up3 grab],g','showplot',true);

figure

dgstruct=svmtrain([down grab],g','showplot',true);

5. MATLAB CODE FOR NEURAL NETWORK TRAINING

functionnet=nntrain1()

P=xlsread('D:\MATLAB\R2009a\work\Sound Card\Database\neuralnetwork.xls');

T=xlsread('D:\MATLAB\R2009a\work\Sound Card\Database\target.xls');

eg=0;

sc=.00009;

net=newrb(P,T,eg,sc);

6. MATLAB CODE FOR SVM CLASSIFICATION

Function ysvm=justclassify(dustruct,u12struct,uu1struct,u2ustruct,

ugstruct,dgstruct,sample)

dug=svmclassify(dustruct,sample, 'showplot',false);

u12g=svmclassify(u12struct,sample, 'showplot',false);

uu1g=svmclassify(uu1struct,sample, 'showplot',false);


83/91

69

u2ug=svmclassify(u2ustruct,sample, 'showplot',false);

ugg=svmclassify(ugstruct,sample, 'showplot',false);

dgg=svmclassify(dgstruct,sample, 'showplot',false);

if(sample(1)sample(2))

display('SVM : UP 2 MOVEMENT')

ysvm=2;

elseif(u2ug==1 & uu1g==-1& sample(1)>sample(2) & ugg==-1)


ysvm=3;

elseif(u12g==-1 & uu1g==1& sample(1)>sample(2))


ysvm=1;

elseif(ugg==1 & dgg==1)

display('SVM : GRAB MOVEMENT')

ysvm=4;

end

7. MATLAB CODE FOR KNN CLASSIFICATIONfunction yknn=knncls(Sample,up1,up2,up3,down,grab)

fori=1:13

distup1(i)=Eucldist(Sample,up1(:,i));



distdown(i)=Eucldist(Sample,down(:,i));

distgrab(i)=Eucldist(Sample,grab(:,i));

end

up1dist=sort(distup1);



downdist=sort(distdown);

grabdist=sort(distgrab);

fori=1:3

dup1=sum(up1dist(i));



ddown=sum(downdist(i));


84/91

70

dgrab=sum(grabdist(i));

end

d=min(min(min(dup1,dup2),min(dup3,ddown)),dgrab);

if(d==dup1)

display('KNN : UP 1 MOVEMENT');yknn=1;

elseif(d==dup2)

display('KNN : UP 2 MOVEMENT');

yknn=2;

elseif(d==dup3)

display('KNN : UP 3 MOVEMENT');

yknn=3;

elseif(d==ddown)

display('KNN : DOWN MOVEMENT');yknn=5;

elseif(d==dgrab)

display('KNN : GRAB MOVEMENT');

yknn=4;

else

display('KNN : NO MOVEMENT');

end

end

end

end

8. MATLAB CODE FOR NEURAL NETWORK CLASSIFICATION

functionynn=nnout1(net,sample)

z=sim(net,sample);

[c,i]=max(z);

if(i==1)

display('ANN : UP 3 MOVEMENT')

ynn=3;

end

if(i==2)


ynn=2;

end

if(i==3)


ynn=1;

end


85/91

71

if(i==4)

display('ANN : DOWN MOVEMENT')

ynn=5;

end

if(i==5)display('ANN : GRAB MOVEMENT')

ynn=4;

end

9. MATLAB CODE FOR CLASSIFICATION

function[y,k]=justclassifier()

k=1;

while(1)

[x1,x2,nom1,nom2]=emgtestlatest();

if and(max(x1)nom2)

f1=(norm(x1)/sqrt(length(x1)));f2=(norm(x2)/sqrt(length(x1)));

sample=[f1 f2];

tic

ysvm=justclassify(dustruct,u12struct,uu1struct,u2ustruct,ugstruct,dgstruct,

sample);

toc

tic

yknn=knncls(sample,up1,up2,up3,down,grab);

toc

tic

ynn1=nnout1(net1,sample');

toc

if(ysvm==yknn || ysvm==ynn1)

y=ysvm;

end

if(yknn==ynn || yknn==ynn1)

y=yknn;


86/91

72

end

end

ify==4

k=not(k);

endpause(2);

end

10. MATLAB CODE FOR SENDING DATA TO MANIPULATOR

function sendata(y,k,s)

global s;

fopen(s);if y==0

fwrite(s,'N','char');

end

if y==1

fwrite(s,'A','char');

end

human computer interface using emg signals - hand gesture based manipulator control

Documents