gesture-aware remote controls: guidelines and interaction techniques
DESCRIPTION
TRANSCRIPT
![Page 1: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/1.jpg)
GESTURE-AWARE REMOTE CONTROLS:GUIDELINES & INTERACTION TECHNIQUES
Gilles Bailly, Dong-Bach Vo, Eric Lecolinet and Yves GuiardTelecom ParisTech, LTCI - CNRS
mardi 15 novembre 2011
![Page 2: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/2.jpg)
Dong-Bach Vo ‒ ICMI 20112
More and more services and contents:• Electronic program guide
• Video on demand
• Web based widgets
• Web applications
• Personal content
Interactive television
mardi 15 novembre 2011
![Page 3: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/3.jpg)
Dong-Bach Vo ‒ ICMI 2011
More and more services and contents:• Electronic program guide
• Video on demand
• Web based widgets
• Web applications
• Personal content
3
Interactive television
mardi 15 novembre 2011
![Page 4: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/4.jpg)
Dong-Bach Vo ‒ ICMI 20114
More and more services and contents:• Electronic program guide
• Video on demand
• Web based widgets
• Web applications
• Personal content
Interactive television
mardi 15 novembre 2011
![Page 5: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/5.jpg)
Dong-Bach Vo ‒ ICMI 20115
More and more services and contents:• Electronic program guide
• Video on demand
• Web based widgets
• Web applications
• Personal content
Interactive television
mardi 15 novembre 2011
![Page 6: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/6.jpg)
Dong-Bach Vo ‒ ICMI 20116
More and more services and contents:• Electronic program guide
• Video on demand
• Web based widgets
• Web applications
• Personal content
Interactive television
mardi 15 novembre 2011
![Page 7: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/7.jpg)
Dong-Bach Vo ‒ ICMI 2011
Inspired from computer interaction:• Deep hierarchical menus
• Long lists
7
Interacting with interactive television
mardi 15 novembre 2011
![Page 8: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/8.jpg)
Dong-Bach Vo ‒ ICMI 20118
Interacting with interactive television
Inspired from computer interaction:• Deep hierarchical menus
• Long lists
mardi 15 novembre 2011
![Page 9: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/9.jpg)
Dong-Bach Vo ‒ ICMI 20119
Interacting with interactive television
Inspired from computer interaction:• Deep hierarchical menus
• Long lists
mardi 15 novembre 2011
![Page 10: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/10.jpg)
Dong-Bach Vo ‒ ICMI 201110
Inspired from computer interaction:• Deep hierarchical menus
• Long lists
Interacting with interactive television
mardi 15 novembre 2011
![Page 11: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/11.jpg)
Dong-Bach Vo ‒ ICMI 2011
Problems:• Browsing
• Commands
11
Remote control
mardi 15 novembre 2011
![Page 12: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/12.jpg)
Dong-Bach Vo ‒ ICMI 2011
Problems:• Browsing
• Commands
12
Remote control
mardi 15 novembre 2011
![Page 13: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/13.jpg)
Dong-Bach Vo ‒ ICMI 201113
Problems:• Browsing
• Commands
Remote control
mardi 15 novembre 2011
![Page 14: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/14.jpg)
Dong-Bach Vo ‒ ICMI 201114
Problems:• Browsing
• Commands
Remote control
Limited expressiveness
mardi 15 novembre 2011
![Page 15: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/15.jpg)
Dong-Bach Vo ‒ ICMI 2011
Improving expressiveness
Solution?• Add a new button for each new
command
15
mardi 15 novembre 2011
![Page 16: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/16.jpg)
Dong-Bach Vo ‒ ICMI 201116
Improving expressiveness
Pultius TV remote control [Artlebedev’07]
Solution?• Add a new button for each new
command
mardi 15 novembre 2011
![Page 17: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/17.jpg)
Dong-Bach Vo ‒ ICMI 201117
Can we benefit from moving the remote control
in 3D space to improve expressiveness?
Solution?• Add a new button for each new
command
Improving expressiveness
mardi 15 novembre 2011
![Page 18: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/18.jpg)
Dong-Bach Vo ‒ ICMI 2011
Approach
Advantages of 3D gestures:• Allow large vocabulary of gestures
• Can be easy to remember
• Free visual focus from the remote control
• Comfortable to perform gestures at home [Rico et al.’10]
• Remote controls can remain small and easy to hold
18
mardi 15 novembre 2011
![Page 19: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/19.jpg)
Dong-Bach Vo ‒ ICMI 2011
Approach
Advantages of 3D gestures:• Allow large vocabulary of gestures
• Can be easy to remember
• Free visual focus from the remote control
• Comfortable to perform gestures at home [Rico et al.’10]
• Remote controls can remain small and easy to hold
18
mardi 15 novembre 2011
![Page 20: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/20.jpg)
Dong-Bach Vo ‒ ICMI 201119
1. Interactive television2. Related work3. Observing spontaneous gestures 4. Exploring new gesture dimension5. Designing new interaction techniques6. Conclusion
mardi 15 novembre 2011
![Page 21: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/21.jpg)
Dong-Bach Vo ‒ ICMI 2011
Related work
20
3D gestures offer 6 degrees of freedom:• 3 translations
• 3 rotations
X
Y
Z
mardi 15 novembre 2011
![Page 22: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/22.jpg)
Dong-Bach Vo ‒ ICMI 2011
Related work
21
3D gestures offer 6 degrees of freedom:• 3 translations
• 3 rotations
Pitch
Yaw
Roll
Z
X
Y
mardi 15 novembre 2011
![Page 23: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/23.jpg)
Dong-Bach Vo ‒ ICMI 2011
Related work
22
Motion marking menu system [Oakley’03]
Pitch X
Rotations:• 1 dimension
• 2 dimensions
• Discretization of tilt angle with visual feedback [Rahman et al.’09]
mardi 15 novembre 2011
![Page 24: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/24.jpg)
Dong-Bach Vo ‒ ICMI 2011
Related work
23
GesText [Jones et al.’10]
Pitch
Roll Z
X
Rotations:• 1 dimension
• 2 dimensions
• Discretization of tilt angle with visual feedback [Rahman et al.’09]
mardi 15 novembre 2011
![Page 25: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/25.jpg)
Dong-Bach Vo ‒ ICMI 2011
Related work
24
Yaw
Roll
Z
Y
12 levels
16 levels
Rotations:• 1 dimension
• 2 dimensions
• Discretization of tilt angle with visual feedback [Rahman et al.’09]
mardi 15 novembre 2011
![Page 26: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/26.jpg)
Dong-Bach Vo ‒ ICMI 201125
1. Interactive television2. Related work3. Observing spontaneous gestures 4. Exploring new gesture dimension5. Designing new interaction techniques6. Conclusion
mardi 15 novembre 2011
![Page 27: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/27.jpg)
Dong-Bach Vo ‒ ICMI 2011
Experiment 1: spontaneous gestures
What gestures users perform spontaneously for moving targets?• Which degree(s) of freedom?
• Is it context dependant (smartphone vs. remote controls)?
26
Pitch
Yaw
Roll
Z
X
Y
mardi 15 novembre 2011
![Page 28: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/28.jpg)
Dong-Bach Vo ‒ ICMI 2011
Task:• Perform the gesture that would
move the target to this position
Procedure:• Smartphone vs. Remote control
• Observation from video recordings
27
Experiment 1: spontaneous gestures
mardi 15 novembre 2011
![Page 29: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/29.jpg)
Dong-Bach Vo ‒ ICMI 2011
Task:• Perform a gesture which has
produced this effect
Procedure:• Smartphone vs. Remote control
• Observation from video recordings
28
Experiment 1: spontaneous gestures
mardi 15 novembre 2011
![Page 30: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/30.jpg)
Dong-Bach Vo ‒ ICMI 2011
Results:• Either rotations or translations
• 14 of 18 users perform rotations
• Pitch and Roll only in smartphone condition
• Pitch and Yaw in remote control condition
• Roll can be used for something else
29
0123456789
(S) Condition (RC) Condition
Num
ber o
f partic
ipan
ts
Pitch & Roll rotations
Pitch & Yaw rotations
X & Y translations
Experiment 1: spontaneous gestures
mardi 15 novembre 2011
![Page 31: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/31.jpg)
Dong-Bach Vo ‒ ICMI 2011
Results:• Either rotations or translations
• 14 of 18 users perform rotations
• Pitch and Roll only in smartphone condition
• Pitch and Yaw in remote control condition
• Roll can be used for something else
30
0123456789
(S) Condition (RC) Condition
Num
ber o
f partic
ipan
ts
Pitch & Roll rotations
Pitch & Yaw rotations
X & Y translations
Experiment 1: spontaneous gestures
Pitch
RollZ
Xmardi 15 novembre 2011
![Page 32: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/32.jpg)
Dong-Bach Vo ‒ ICMI 2011
Results:• Either rotations or translations
• 14 of 18 users perform rotations
• Pitch and Roll only in smartphone condition
• Pitch and Yaw in remote control condition
• Roll can be used for something else ntrol condition
31
0123456789
(S) Condition (RC) Condition
Num
ber o
f partic
ipan
ts
Pitch & Roll rotations
Pitch & Yaw rotations
X & Y translations
Experiment 1: spontaneous gestures
Pitch
Yaw
X
Y
mardi 15 novembre 2011
![Page 33: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/33.jpg)
Dong-Bach Vo ‒ ICMI 2011
Results:• Either rotations or translations
• 14 of 18 users perform rotations
• Pitch and Roll only in smartphone condition
• Pitch and Yaw in remote control condition
• Roll can be used for something else
32
0123456789
(S) Condition (RC) Condition
Num
ber o
f partic
ipan
ts
Pitch & Roll rotations
Pitch & Yaw rotations
X & Y translations
Experiment 1: spontaneous gestures
Pitch
Yaw
X
Y
RollZ
mardi 15 novembre 2011
![Page 34: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/34.jpg)
Dong-Bach Vo ‒ ICMI 201133
1. Interactive television2. Related work3. Observing spontaneous gestures 4. Exploring roll gestures5. Designing new interaction techniques6. Conclusion
mardi 15 novembre 2011
![Page 35: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/35.jpg)
Dong-Bach Vo ‒ ICMI 2011
Experiment 2: exploring roll gestures
What we know:• 16 items selection with visual feedback [Rahman’09]
• 3 items selection in eyes free selection [Oakley’07]
34
mardi 15 novembre 2011
![Page 36: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/36.jpg)
Dong-Bach Vo ‒ ICMI 201135
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 37: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/37.jpg)
Dong-Bach Vo ‒ ICMI 201136
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 38: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/38.jpg)
Dong-Bach Vo ‒ ICMI 201137
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Step 1: Roll to prepare the gesture
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 39: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/39.jpg)
Dong-Bach Vo ‒ ICMI 201137
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Step 1: Roll to prepare the gesture
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 40: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/40.jpg)
Dong-Bach Vo ‒ ICMI 201138
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Step 2: Press the trigger
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 41: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/41.jpg)
Dong-Bach Vo ‒ ICMI 201139
Question: How many levels users can control without visual feedback?
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Step 3: Roll gesture
mardi 15 novembre 2011
![Page 42: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/42.jpg)
Dong-Bach Vo ‒ ICMI 201140
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Step 4: Release the trigger
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 43: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/43.jpg)
Dong-Bach Vo ‒ ICMI 201141
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 44: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/44.jpg)
Dong-Bach Vo ‒ ICMI 201142
Task:Use any roll amplitude you need to reach the target
• Roll - Press - Roll - Release strategy
• No feedback provided
Procedure:• 9 participants
• Items set size (5, 7, 9, 11 items)
Experiment 2: exploring roll gestures
Question: How many levels users can control without visual feedback?
mardi 15 novembre 2011
![Page 45: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/45.jpg)
Dong-Bach Vo ‒ ICMI 2011
Angular range:• Differs largely among
participants
• Depends on item-set size
Angular variation:• Constant variation between two
contiguous items
43
162° 326°
Minimum and maximum angular range for 7 items
Experiment 2: results
mardi 15 novembre 2011
![Page 46: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/46.jpg)
Dong-Bach Vo ‒ ICMI 2011
Angular range:• Differs largely among
participants
• Depends on item-set size
Angular variation:• Constant variation between two
contiguous items
44
Experiment 2: results
0
100
200
300
400
0 1 2 3 4 6 7 8 9 10 11∆
angle (in
deg
rees
)
number of items in set
mardi 15 novembre 2011
![Page 47: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/47.jpg)
Dong-Bach Vo ‒ ICMI 2011
Angular range:• Differs largely among
participants
• Depends on item-set size
Angular variation:• Constant variation between two
contiguous items
45
Experiment 2: results
-200-160-120-80-4004080120160200
-3 -2 -1 0 1 2 3
Mean angular varation for 7 items
RightLeft Center
mardi 15 novembre 2011
![Page 48: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/48.jpg)
Dong-Bach Vo ‒ ICMI 201146
Experiment 2: results
Recognition:• KNN algorithm
• euclidian distance on angular variation
• cross-validation technique
Results:• Very good for 5 items (96.3%)
• Insufficient for 9 and 11 items
• 7 items: • 87.7% for cross-validation
• 96% with user specific training
mardi 15 novembre 2011
![Page 49: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/49.jpg)
Dong-Bach Vo ‒ ICMI 201147
Experiment 2: results
0
25
50
75
100
5 items 7 items 9 items 11 itemsac
curacy
(in %)
mean accuracy rate
96.3Recognition:• KNN algorithm
• euclidian distance on angular variation
• cross-validation technique
Results:• Very good for 5 items (96.3%)
• Insufficient for 9 and 11 items
• 7 items: • 87.7% for cross-validation
• 96% with user specific training
mardi 15 novembre 2011
![Page 50: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/50.jpg)
Dong-Bach Vo ‒ ICMI 201148
Experiment 2: results
0
25
50
75
100
5 items 7 items 9 items 11 itemsac
curacy
(in %)
mean accuracy rate
77.1
63.1
Recognition:• KNN algorithm
• euclidian distance on angular variation
• cross-validation technique
Results:• Very good for 5 items (96.3%)
• Insufficient for 9 and 11 items
• 7 items: • 87.7% for cross-validation
• 96% with user specific training
mardi 15 novembre 2011
![Page 51: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/51.jpg)
Dong-Bach Vo ‒ ICMI 201149
Experiment 2: results
Recognition:• KNN algorithm
• euclidian distance on angular variation
• cross-validation technique
Results:• Very good for 5 items (96.3%)
• Insufficient for 9 and 11 items
• 7 items: • 87.7% for cross-validation
• 96% with user specific training
0
25
50
75
100
5 items 7 items 9 items 11 itemsac
curacy
(in %)
mean accuracy rate
87.7
mardi 15 novembre 2011
![Page 52: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/52.jpg)
Dong-Bach Vo ‒ ICMI 201150
1. Interactive television2. Related work3. Observing spontaneous gestures 4. Exploring roll gestures5. Designing new interaction techniques6. Conclusion
mardi 15 novembre 2011
![Page 53: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/53.jpg)
Dong-Bach Vo ‒ ICMI 2011
Designing interaction techniques
51
Goals:• two-level circular menus
• that combine gestural modalities
• Pitch & Yaw
• Roll
• Directional pad buttons
mardi 15 novembre 2011
![Page 54: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/54.jpg)
Dong-Bach Vo ‒ ICMI 2011
Designing interaction techniques
52
Design space:• 1 modality per menu
level
• D-pad used as a baseline
• Unpromising techniques dismissed at pretest
mardi 15 novembre 2011
![Page 55: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/55.jpg)
Dong-Bach Vo ‒ ICMI 2011
Designing interaction techniques
53
Design space:• 1 modality per menu
level
• D-pad used as a baseline
• Unpromising techniques dismissed at pretest
2nd Level
1st LevelPitch & Yaw Roll Directional Pad
Pitch & Yaw
Roll
Directional Pad
mardi 15 novembre 2011
![Page 56: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/56.jpg)
Dong-Bach Vo ‒ ICMI 2011
Designing interaction techniques
54
Design space:• 1 modality per menu
level
• D-pad used as a baseline
• Unpromising techniques dismissed at pretest
2nd Level
1st LevelPitch & Yaw Roll Directional Pad
Pitch & Yaw
Roll
Directional Pad Baseline
mardi 15 novembre 2011
![Page 57: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/57.jpg)
Dong-Bach Vo ‒ ICMI 2011
Designing interaction techniques
55
Design space:• 1 modality per menu
level
• D-pad used as a baseline
• Unpromising techniques dismissed at pretest
2nd Level
1st LevelPitch & Yaw Roll Directional Pad
Pitch & Yaw
Roll
Directional Pad
Gesture trigger
penalty
Biomechanical
constraint
Poor results in
pretests
Gesture trigger
penalty
Baseline
mardi 15 novembre 2011
![Page 58: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/58.jpg)
Dong-Bach Vo ‒ ICMI 2011
2nd Level
1st LevelPitch & Yaw Roll Directional Pad
Pitch & Yaw
Roll
Directional Pad
3D multi-strokes
menu
3D gestures
parallelization
Gesture trigger
penalty
Biomechanical
constraint
Poor results in
pretests
Gesture trigger
penalty
2D/3D gestures
parallelization
2D/3D gestures
parallelizationBaseline
56
Designing interaction techniques
Design space:• 1 modality per menu
level
• D-pad used as a baseline
• Unpromising techniques dismissed at pretest
• 5 techniques to evaluate
mardi 15 novembre 2011
![Page 59: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/59.jpg)
Dong-Bach Vo ‒ ICMI 2011
Procedure:• Menu with 4 * 4 items
• Combination of modalities
• User feedback
• 13 participants
Task:• Perform the gesture
corresponding to the symbolic stimulus
57
Experiment 3: interaction techniques
mardi 15 novembre 2011
![Page 60: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/60.jpg)
Dong-Bach Vo ‒ ICMI 201158
0
20,0
40,0
60,0
80,0
100,0
Pad/Pad Pad/P&Y Pad/Roll
Mean accuracy rate by technique
Accu
racy
in %
Best modality combinations for accuracy:• Pad / Pad (baseline)
• Pad / Pitch & Yaw (better than baseline)
• Pad / Roll
Experiment 3: results
97.4 98.4 94.2
mardi 15 novembre 2011
![Page 61: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/61.jpg)
Dong-Bach Vo ‒ ICMI 201159
0
425
850
1275
1700
Pad/Pad Pad/P&Y Pad/Roll
Mean time selection by technique
Tim
e in m
s
Best modality combinations for speed:• Similar results
• Pad / Roll is a bit slower
Experiment 3: results
1400 13901650
mardi 15 novembre 2011
![Page 62: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/62.jpg)
Dong-Bach Vo ‒ ICMI 201160
User satisfaction:• Pad / Pad
• then Pad / Pitch & Yaw
• then P&Y / P&Y
• then Pad / Roll
Experiment 3: results
01234567
Speed Accuracy Pleasant Tiring Easy to learn
Subjective preferences
Pad/Pad Pad/P&Y Pad/Roll P&Y/P&Y
mardi 15 novembre 2011
![Page 63: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/63.jpg)
Dong-Bach Vo ‒ ICMI 2011
Multimedia system menu
61
Novice mode:• 2-level circular menu
• 2 modalities
• Novice mode:• 1st level : Pitch and Yaw
• 2nd level : Roll
mardi 15 novembre 2011
![Page 64: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/64.jpg)
Dong-Bach Vo ‒ ICMI 2011
Multimedia system menu
62
Novice mode:• 2-levels circular menu
• 1st level selection
• 2nd level selection
mardi 15 novembre 2011
![Page 65: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/65.jpg)
Dong-Bach Vo ‒ ICMI 201163
Multimedia system menu
Novice mode:• 2-levels circular menu
• 1st level selection
• 2nd level selection
mardi 15 novembre 2011
![Page 66: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/66.jpg)
Dong-Bach Vo ‒ ICMI 201164
Expert mode:• Same gestures
• But the menu does not appear
• Implicit learning of gestures
Multimedia system menu
mardi 15 novembre 2011
![Page 67: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/67.jpg)
Dong-Bach Vo ‒ ICMI 201165
1. Interactive television2. Related work3. Observing spontaneous gestures 4. Exploring roll gestures5. Designing new interaction techniques6. Conclusion
mardi 15 novembre 2011
![Page 68: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/68.jpg)
Dong-Bach Vo ‒ ICMI 2011
Conclusion
66
What we have learned:• Experiment 1:
• Users rather use rotations when interacting with handheld devices
• With a remote control, they use pitch and yaw to interact with a distant screen
• Experiment 2: Roll gestures without feedback work fine:
• for 5 levels
• for 7 levels with user specific training
• Using button and 3D gestures can efficiently augment remote control expressiveness
mardi 15 novembre 2011
![Page 69: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/69.jpg)
Dong-Bach Vo ‒ ICMI 2011
Conclusion
67
What we have learned:• Experiment 3:
• 3D gestures provide good performance
• especially when combined with d-pad buttons
• Conclusion:• Moving the remote control into space is indeed an effective
way to improve expressiveness
mardi 15 novembre 2011
![Page 70: Gesture-aware remote controls: guidelines and interaction techniques](https://reader034.vdocuments.us/reader034/viewer/2022042713/54882191b479590f0d8b559f/html5/thumbnails/70.jpg)
Dong-Bach Vo ‒ ICMI 2011
Thank you for your attention!
http://perso.telecom-paristech.fr/~via/[email protected]
GESTURE-AWARE REMOTE CONTROLS:GUIDELINES & INTERACTION TECHNIQUES
mardi 15 novembre 2011