cindy song sharena paripatyadar. use vision for hci determine steps necessary to incorporate vision...
Post on 21-Dec-2015
214 views
TRANSCRIPT
• Use vision for HCI • Determine steps necessary to
incorporate vision in HCI applications• Examine concerns & implications of
such applications
Background and TrendsBackground and Trends In Today’s world:
• Many devices with integrated cameras• Many personal webcams
Our Goal:• To understand how to take advantage of
these one camera systems
Freeman and Roth For hand posture
analysis Creates histograms
of local orientation using feature vectors from pixel intensity
Recognizes 10 gestures in real time
Triesch and von der Malsburg
Based on Elastic Graph Matching
Extended for skin color feature detection
Recognizes 12 gestures
Freeman Uses one open
hand to control onscreen display
Real time application
Hand may not be prominent in image
5 participants, various technical backgrounds, age 20-27
Using computer with remote control Used alternate monitor to show user
video captured
Small set of user-intuitive gestures are easy to remember, but need some menu reminder
Show rationale behind gestures Visual feedback to show recognized
command before execution Concerns with:
Low-light conditionCamera field of view Webcam configurationResponsivenessAccuracy
ProsDon’t have to search for remoteDon’t have to touch remote while eatingNo battery to run down
ConsDoesn’t have as many features as remoteDoesn’t work in dark environmentsMore ambiguous than remote, more errors
possible – know what each button will do
Calibration• Short training based on person’s hand and
lighting conditions - < 1 sec needed• Determines correct lighting & with skin color
data• Learns specific hand features
Hand Location• Determines hand position in image using
skin color• Fill in missing portions of hand • Create bounding box
Pattern Recognition• Created set patterns based on 10
gestures• Counts number of finger regions for
gestures 1-5• For gestures 6-10, based on number
regions detected, looks at other patterns i.e. for 6 determine ratio of finger width to
space between fingers
Gesture Determination• 20 frames needed to recognize the gesture• Avoids recognizing accidental gestures
Complex Backgrounds • First skin color analysis• Then find large connected regions of fingers
and hand
Motion • Static gestures & frame by frame analysis• Allow for moving camera• Gesture determination corrects obscurities
or out of frame hand positioning
5 participants, various technical backgrounds, age 20-27
Taught users 2-4 gestures Quizzed users on gestures learned Ran gesture recognition algorithm to
provide feedback Asked several follow up questions
Useful for learning sign language, teaching kids to count
Instant feedback necessary Nice to know how to correct gesture Needs high accuracy Other applications
• Some said Media Player application more useful
• Or use as security system (hand gestures as a password)
Of implementationReal time is difficultPattern recognition for specific gestures vs.
technique for all types of gesturesComplex/moving backgrounds important for
real world applications
Of user studiesVideo is valuable avenue for many applicationsAccuracy and responsiveness are importantIn one camera systems, there is a tradeoff
between convenience and clarity