finger gesture recognition through sweep sensor
DESCRIPTION
Finger Gesture Recognition through Sweep Sensor. Pong C Yuen 1 , W W Zou 1 , S B Zhang 1 , Kelvin K F Wong 2 and Hoson H S Lam 2 1 Department of Computer Science Hong Kong Baptist University 2 World Fair International Ltd. Outline. Motivations Design Criteria Proposed Method - PowerPoint PPT PresentationTRANSCRIPT
Finger Gesture Recognition through
Sweep SensorPong C Yuen1, W W Zou1, S B Zhang1, Kelvin K F Wong2 and Hoson H S
Lam2
1Department of Computer ScienceHong Kong Baptist University
2World Fair International Ltd
OutlineMotivations
Design Criteria
Proposed Method
Experimental results
Conclusions
MotivationsVision-based
interfaceInsert some images using face, expression, body movement…
Sensor-based interface
http://www.blogcdn.com/www.tuaw.com/media/2008/11/mac-101_-multi-touch-tips.jpg
http://www.fitbuff.com/wp-content/uploads/2007/10/wii-fitness.jpg
There should be a video about the body movement interface
Common objective: natural input to replace traditional physical input devices
MotivationsWhile many sensor-based gesture input have been developed, there is no algorithm/system using sweep sensor
Why Sweep Sensor?low costNo latency problem (fingerprint recognition)popularity
Design CriteriaUser friendliness
easily performed by a user. intuitive and easy to understand.
User independentGeneric for all users.
Robustnessdiversity of patterns captured.
Efficiency Real-time applicationmobile devices
Classification
t > t0
left rightNo
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Feature Extractionnoise
reduction
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
SegmentationInput image
CharacteristicsFormulate the noise
Proposed MethodInput image
CharacteristicsFormulate the noise
Segmentation
Feature Extractionnoise
reduction
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
Classification
t > t0
left rightNo
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Input image
CharacteristicsFormulate the noise
Segmentation
Feature Extractionnoise
reduction
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
Classification
t > t0
left rightNo
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Input Image Characteristics
Different sensor characteristicsNoise level
Figure 2. The block diagram of feature extraction
SegmentationOwing to different sensor characteristics, the gesture images obtained, even the gesture is the same, will be different
Segmentation by estimating the sweeping time
noise reduction
vertical gradient thresholding horizontal
projection
Segmentation (cont.)
blank partsw
eeping part
noise reduction
vertical gradient
THthresholdin
g
horizontal projection
sth1t 't
] 0[on )( ), ))((1 ( 1)(0
2
1
1
ttEdttstcth ts
t
s
Feature ExtractionTime information t (sweeping time)
Finger motion information d (direction)Left and rightLeft diagonal and right diagonal
Feature Extraction (left / right)
noise reduction
Left
Right
direction
enhancement
input image
direction estimation
direction index D = Pleft - Pright
i-th fingerprint texture
A B
C D )(
)(
CBDAP
CBDAP
right
left
Feature Extraction (left tick / right tick)
noise reduction
envelope enhancement
input image
direction estimation
direction index D = Dleft /Dright
0 40 80 120 160 2000
20
40
60
80
100
120
140
160
0 40 80 120 160 2000
20
40
60
80
100
120
140
160
0
0
miiright
miileft
yD
yD
envelope
yi
i
y
ClassificationA very simple rule based on a combination of movement
Classification tree (decision tree)
left rightNo
left tick right tick
Yes
feature vector t > t0
D > 1.3 left tick
right tickD > 1/1.3
D > 0.5 left
rightD < -0.5
Designed GesturesLeft and Right Left tick and Right
tick
Experimental Results2 testing groups
3 technical users – Engineers, and technical managers, research staff (95.0%)3 Non-technical users – secretary, clerk (86.87%)
Test on different sensors4 different sensors manufacture at different period of time
Experimental ResultsEvaluation interface
There should be a video here
Experimental Results
Sensor1 Sensor2 Sensor3 Sensor40
102030405060708090
100
Results of fingerprint recognition using different sensors
success fail%
Results by 3 non-technical staff with 4 different sensors
Experimental ResultsIntegrated application with an image viewer
There should be a video here
Conclusions
Thank You