finger gesture recognition through sweep sensor pong c yuen 1, w w zou 1, s b zhang 1, kelvin k f...
TRANSCRIPT
Finger Gesture Recognition through
Sweep Sensor
Pong C Yuen1, W W Zou1, S B Zhang1, Kelvin K F Wong2 and Hoson H S Lam2
1Department of Computer ScienceHong Kong Baptist University
2World Fair International Ltd
Outline
Motivations
Design Criteria
Proposed Method
Experimental results
Conclusions
MotivationsVision-based
interfaceInsert some images using face, expression, body movement…
Sensor-based interface
http://www.blogcdn.com/www.tuaw.com/media/2008/11/mac-101_-multi-touch-tips.jpg
http://www.fitbuff.com/wp-content/uploads/2007/10/wii-fitness.jpg
There should be a video about the body movement interface
Common objective: natural input to replace traditional physical input devices
Motivations
While many sensor-based gesture input have been developed, there is no algorithm/system using sweep sensor
Why Sweep Sensor?low cost
No latency problem (fingerprint recognition)
popularity
Design CriteriaUser friendliness
easily performed by a user.
intuitive and easy to understand.
User independentGeneric for all users.
Robustnessdiversity of patterns captured.
Efficiency Real-time application
mobile devices
Classification
t > t0
left right
No
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Feature Extraction
noise reductio
n
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
SegmentationInput image
CharacteristicsFormulate the noise
Proposed MethodInput image
CharacteristicsFormulate the noise
Segmentation
Feature Extraction
noise reductio
n
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
Classification
t > t0
left right
No
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Input image
CharacteristicsFormulate the noise
Segmentation
Feature Extraction
noise reductio
n
envelope enhanceme
ntinput image
direction estimation
direction index D = Dleft
/Dright
envelope
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
0 40 80 120 160 200
0
20
40
60
80
100
120
140
160
yi
i
y
0
0
miiright
miileft
yD
yD
Classification
t > t0
left right
No
left tick right tick
Yes
feature vector
D > 0.5 left
right
D > 1.3 left tick
right tickD > 1/1.3
D < -0.5
Input Image Characteristics
Different sensor characteristicsNoise level
Figure 2. The block diagram of feature extraction
Segmentation
Owing to different sensor characteristics, the gesture images obtained, even the gesture is the same, will be different
Segmentation by estimating the sweeping time
noise reduction
vertical gradient
thresholdinghorizontal projection
Segmentation (cont.)
blank partsw
eeping part
noise reduction
vertical gradient
THthresholdin
g
horizontal projection
sth1t 't
] 0[on )( ), ))((1 ( 1)(
0
2
1
1
ttEdttst
cth ts
t
s
Feature Extraction
Time information t (sweeping time)
Finger motion information d (direction)Left and right
Left diagonal and right diagonal
Feature Extraction (left / right)
noise reduction
Left
Right
direction
enhancement
input image
direction estimation
direction index D = Pleft - Pright
i-th fingerprint texture
A B
C D )(
)(
CBDAP
CBDAP
right
left
Feature Extraction (left tick / right
tick)
noise reduction
envelope enhancement
input image
direction estimation
direction index D = Dleft /Dright
0 40 80 120 160 2000
20
40
60
80
100
120
140
160
0 40 80 120 160 2000
20
40
60
80
100
120
140
160
0
0
miiright
miileft
yD
yD
envelope
yi
i
y
Classification
A very simple rule based on a combination of movement
Classification tree (decision tree)
left rightNo
left tick right tick
Yes
feature vector
t > t0
D > 1.3 left tick
right tick
D > 1/1.3
D > 0.5 left
right
D < -0.5
Designed Gestures
Left and RightLeft tick and Right
tick
Experimental Results
2 testing groups3 technical users – Engineers, and technical managers, research staff (95.0%)
3 Non-technical users – secretary, clerk (86.87%)
Test on different sensors4 different sensors manufacture at different period of time
Experimental Results
Evaluation interface
There should be a video here
Experimental Results
Sensor1 Sensor2 Sensor3 Sensor40
10
20
30
40
50
60
70
80
90
100
Results of fingerprint recognition using different sensors
success fail
%
Results by 3 non-technical staff with 4 different sensors
Experimental ResultsIntegrated application with an image viewer
There should be a video here
Conclusions
Thank You