should i decide or let the machine decide for me? explorations in the use, misuse and disuse of...
TRANSCRIPT
Should I Decide or Let the Machine Decide for Me? Explorations in the
Use, Misuse and Disuse of Automation
The Soldier Study Prancers
Skip, Mary, Linda, Teresa, Chris, Jamie, Kelly, Marielle, Tommy, Andy, Becca, Leslie,
Julie, Nick and Greg
Examples of Automation Versus Nonautomated Options
• Pilot uses automated or manual guidance systems to fire missiles.
• Accountant uses software program or personal knowledge to compute taxes.
• Battlefield Combat Identification System
• Power Mower or Push Mower
Goals of Automation Use
• Save time, energy or money
• Make an impression
• Practice
• Achieve a performance objective
Optimal Decision Requires
• Assessing the likelihood of success using automation.
• Assessing the likelihood of success using a nonautomated alternative.
• Select the option with the highest probability of success.
Objectives of Study 1
• To determine the extent that persons would deviate from an optimal (rational) decision making strategy.
• To discover if automation misuse or automation disuse is the greater problem on this task.
Design
• 3 (Relative Performance: Inferior, Equal, Superior) x 2 (Feedback: Yes, No) between-subjects.
•Choice to base credit on self or computer as the dependent variable.
Trial by Trail Procedure
• Present photo for .75 seconds.
• Participant responds via mouse indicating if the target was in the photo.
• Contrast detector then scans photo for human form. It attempts to determine if the target is in the photo.
Still More Procedure
• After 200 trials, participants in feedback condition are told how many errors they and the machine made. Participants in the No Feedback condition do not receive this information.
• Participants are either inferior, equal or superior to the contrast detector.
High Noon
• All participants are told 10 more trials will be conducted which will determine the extra credit that they receive.
• At this point, the participant must choose to base extra credit on self or machine.
Persons Basing Extra Credit on Self
0
20
40
60
80
100
% C
hoos
ing
Self
Inferior 100 100
Equal 89 100
Superior 89 100
No Feedback Feedback
Rational Use of Automation
0
20
40
60
80
100%
Rat
iona
l Dec
isio
ns
Inferior 100 100
Superior 11 0
No Feedback Feedback
Objectives of Study 2
• To determine the results of the first study were due to hopes of a “heroic turnabout.”
• To find if the outcomes of Study 1 indicate a bias against using automation.
Procedure and Design
•Same basic task as first study.
•3(Relative Performance: Inferior, Equal, Superior) x 2 (Feedback: Yes, No) between-subjects
•Ten trials randomly selected. Choice to base credit on self or contrast detector.
Persons Basing Extra Credit on Self
0
20
40
60
80
100
% C
hoos
ing
Self
Inferior 100 100
Equal 89 89
Superior 89 67
No Feedback Feedback
Rational Use of Automation
0
20
40
60
80
100%
Rat
iona
l Dec
isio
ns
Inferior 100 100
Superior 11 33
No Feedback Feedback
Objectives of Study 3
• To determine if feedback mitigates the bias against using automation.
• To discover what type of feedback is most effective in reducing this bias.
Forms of Feedback
•Trial by Trial: After each trial participant told if target was in the preceding photo.
• Cumulative: After 200 trials, participant told total errors made by self and contrast detector.
•Prior Results: After 200 trials, participants informed that persons who base extra credit on detector usually obtain more points.
Procedure and Design
•Same basic task as first two studies. Detector superior to all participants.
• 2 (Trial by Trial: Yes, No) x 2 (Cumulative: Yes, No) x 2 (Prior Results: Yes, No) between-subjects.
•Ten trials randomly selected. Choice to base credit on self or contrast detector.
Rational Use of Automation
0
20
40
60
80
100
% R
atio
nal D
ecis
ions
None One Two Three
None 16
One 32
Two 33
Three 64
Objectives of Study 4
• To determine if a combination of training participants to use the “automation algorithm” decreases the bias.
• To discover if a combination of training and feedback if more effective than either training or feedback alone.
Rational Use of Automation
0
20
40
60
80
100
Inf-No T 90 100
Inf-T 95 100
Sup-No T 11 40
Sup-T 0 67
No Feedback Feedback