in-car speech user interfaces and their effects on driver cognitive load
DESCRIPTION
Presentation given at FTW, June 2010.TRANSCRIPT
![Page 1: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/1.jpg)
In-Car Speech User Interfaces and their Effects on Driver Cognitive Load
Andrew Kun
![Page 2: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/2.jpg)
Students and staff @ UNH
Jon Oppelaar
Oskar PalinkoZeljko Medenica Nemanja Memarovic
Alex Shyrokov Puneet Lakhanpal
![Page 3: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/3.jpg)
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
![Page 4: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/4.jpg)
Navigation
![Page 5: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/5.jpg)
Music, phones
![Page 6: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/6.jpg)
Police cars
• Toys vs. tools• Vanguard of in-car
ubicomp
![Page 7: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/7.jpg)
Project54
![Page 8: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/8.jpg)
Studies
• Goal: – Evaluation tool– Improved HCI
![Page 9: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/9.jpg)
Cognitive load
• Measures:– Performance
• Driving• Visual attention
– Physiological• Pupil diameter• Heart rate• Skin conductance
– Subjective • NASA TLX
![Page 10: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/10.jpg)
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
![Page 11: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/11.jpg)
DriveSafety driving simulator• tinyurl.com/p54sim
![Page 12: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/12.jpg)
Driving performance
• Variance:– Lane position– Steering wheel angle– Velocity
• Collisions
![Page 13: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/13.jpg)
Seeingmachines eye tracker• tinyurl.com/eyetracker
![Page 14: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/14.jpg)
Visual attention
• Fixations (> 100 ms): number, timing, etc.• Percent dwell time: looking at road
![Page 15: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/15.jpg)
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
![Page 16: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/16.jpg)
Police radio study: SUI vs. hardware
![Page 17: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/17.jpg)
![Page 18: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/18.jpg)
![Page 19: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/19.jpg)
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
![Page 20: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/20.jpg)
SUI characteristics study
• Secondary task: speech control of radio• 2 x 2 x 2 design:
– SR accuracy: high/low– PTT button: yes/no – ambient recognition– Dialog repair strategy: mis-/non-understanding
![Page 21: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/21.jpg)
![Page 22: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/22.jpg)
Rec. accuracy influences steering
Recognition AccuracyHighLow
Mea
n va
rianc
e of
ste
erin
g w
heel
ang
le
[deg
2]
500
400
300
200
100
0
Error bars: +/- 1 SD
![Page 23: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/23.jpg)
Low acc. + PTT influences lane position
PTTYesNo
Mea
n la
ne p
ositi
on v
aria
nce
[m2]
0.3
0.2
0.1
0.0
Error bars: +/- 1 SD
![Page 24: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/24.jpg)
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
![Page 25: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/25.jpg)
Glove PTT study• Motivation: PTT vs. driving performance• Secondary task: speech control of radio• 2 x 3 x 3 design:
– SR accuracy: high/low– PTT activation: push-hold-release/push-release/no push– PTT button: ambient/fixed/glove
Push-hold-release Push-release No-push
Ambient Fixed Glove Ambient Fixed Glove Ambient Fixed Glove
High
Low
![Page 26: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/26.jpg)
Glove PTT study
![Page 27: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/27.jpg)
Fixed PTT operation
![Page 28: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/28.jpg)
Fixed PTT – glances!
![Page 29: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/29.jpg)
Glove PTT operation
![Page 30: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/30.jpg)
Glove PTT – few glances
![Page 31: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/31.jpg)
Glove PTT transcription
![Page 32: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/32.jpg)
Hand position on wheel [degrees]
![Page 33: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/33.jpg)
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
![Page 34: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/34.jpg)
Navigation study
• Problems:– In-car devices vs.
driving performance and visual attention– Driving performance/visual attention vs.
probability of accident
![Page 35: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/35.jpg)
Navigation study
• With vs. without map:– Time looking at the
road?– Effect on driving
performance?• Voice instruction
for both• 8 male subjects
![Page 36: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/36.jpg)
Driving simulator
![Page 37: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/37.jpg)
1
2
2
3 3
1
Driving scenario
![Page 38: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/38.jpg)
Visual attention
• Fixations (> 100 ms): number, timing, etc.• Percent dwell time: looking at road
![Page 39: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/39.jpg)
Driving performance
• Variance:– Lane position– Steering wheel angle– Velocity
• Collisions
![Page 40: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/40.jpg)
Effect of visual attention?
• Average variances (13 segments)• Cross-correlation peaks:
PDT decrease → variance increase?
![Page 41: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/41.jpg)
![Page 42: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/42.jpg)
Eye tracker camera Rear camera
Camera setup
Front camera
![Page 43: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/43.jpg)
Standard PND
![Page 44: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/44.jpg)
Voice-only PND
![Page 45: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/45.jpg)
Questions
• Influence on visual attention?• If yes, effect on driving performance?
![Page 46: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/46.jpg)
PDT on the outside world
Series186%
88%
90%
92%
94%
96%
98%
90.4%
96.90%
Standard PND Spoken output only PND
PDT
[%]
![Page 47: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/47.jpg)
PDT at world vs. distance
60-80 80-100 100-120 120-140 140-16080
85
90
95
100standard spoken only
distance from previous intersection [m]
PDT
on o
utsi
de w
orld
[%]
![Page 48: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/48.jpg)
PDT at screen vs. distance
60-80 80-100 100-120 120-140 140-160
-5
0
5
10
15
20
distance from previous intersection [m]
PDT
on s
tand
ard
PND
[%]
![Page 49: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/49.jpg)
Questions
• Influence on visual attention?• If yes, effect on driving performance?
![Page 50: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/50.jpg)
Cross-correlation
0 1 2 3 4 5 6 7 80
0.001
0.002
0.003
0.004
0.005
0.006
0.007Lane position
standard
p = 0.05
spoken only
lag [seconds]
Rlp
[met
ers^
2 ]
0 1 2 3 4 5 6 7 80
5
10
15
20
25
30
35Steering wheel angle
standard
p = 0.05
spoken only
lag [seconds]
Rstw
[deg
rees
^2 ]
![Page 51: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/51.jpg)
Cross-correlation, gazes > 200 ms
0 1 2 3 4 5 6 7 80
0.001
0.002
0.003
0.004
0.005
0.006
0.007
Lane position
standard
p = 0.05
spoken only
lag [seconds]
Rlp
[met
ers^
2 ]
0 1 2 3 4 5 6 7 80
5
10
15
20
25
30
35
Steering wheel angle
lag [seconds]
Rstw
[deg
rees
^2 ]
![Page 52: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/52.jpg)
Fixations at standard PND screen
0.1-0.2 0.2-0.3 0.3-0.4 0.4-0.5 0.5-0.6 0.6-0.7 0.7-0.8 0.8-0.90
10
20
30
40
50
60
70
duration [sec]
num
ber o
f fixa
tions
[#]
![Page 53: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/53.jpg)
Navigation study conclusions
• Display: (visual attention)↘
• (visual attention)↘ → (variances↗)
• (long gazes at PND) → (variances↗)• Display viewing: 60% of gazes last >200ms• Collision risk: need testing• But…
![Page 54: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/54.jpg)
I prefer to have a PND screen…
Strongly agree Agree Undecided Disagree Strongly disagree
0
6
1 1
0
![Page 55: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/55.jpg)
Outline
• Ubicomp and cars• Studies:
– Police radio– SUI characteristics– Glove PTT– Navigation
• Conclusion
![Page 56: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/56.jpg)
Design implications
• SUI may improve performance.• SUI details matter (SR accuracy, PTT).• Must earn users trust.• Complicated displays may reduce visual
attention even more.• Small screen? Voice-only may be just fine.
![Page 57: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/57.jpg)
Next?
• “You’re on the right track.”• Complex environments?• Other possibilities e.g. augmented reality.• Safety?
![Page 58: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/58.jpg)
Acknowledgements
• US DOJ (NIJ, COPS)• NSF• Microsoft Research• TellMe
![Page 59: In-Car Speech User Interfaces and their Effects on Driver Cognitive Load](https://reader035.vdocuments.us/reader035/viewer/2022070302/548bdd39b479597e6a8b463f/html5/thumbnails/59.jpg)