ux & measures of engagement

mesures d’engagement

Dr. Alan PopeNASA

UX&

UX & Measures of Engagement

World Usability Day @ HEC: User Engagement and Usability

Montreal, Canada November 2015

AlanPope

NASALangleyResearchCenter

1

2

•  Background of Crew State Monitoring Research at NASA Langley

•  Development of a Method for Evaluating Task Engagement Indices

•  In Progress Crew State Monitoring Research at NASA Langley

3

Real-Time State Estimation in a Flight Simulator Using fNIRS Gateau T, Durantin G, Lancelot F, Scannella S, Dehais F (2015)

PLoS ONE 10(3): e0121279

4

Initial Research Agenda

5

•  Background of Crew State Monitoring Research at NASA Langley

•  Development of a Method for Evaluating Task Engagement Indices

•  In Progress Crew State Monitoring Research at NASA Langley

•  ~70% of incidents and accidents are attributed to human error

Human Error in Aviation Incidents and Accidents

•  A primary goal of NASA’s Aeronautics research focus is to improve the National Airspace System which already has an exceptionally high level of safety.

6

Use of Automation in Aviation and Hazardous States of Awareness (HSAs)

•  Automation plays a significant role in the cockpit –  enables humans to perform beyond normal

abilities (longer shifts, improved control, etc.) •  But can lead to suboptimal psychological

states [Aviation Safety Reporting System (ASRS) database] –  complacency –  boredom –  diminished alertness –  compromised vigilance –  lapsing attention –  preoccupation –  absorption 7

8

9

Brainwave Correlates of Hazardous States In

Advanced Concepts Flight Simulator 10

11

•  Background of Crew State Monitoring Research at NASA Langley

•  Development of a Method for Evaluating Task Engagement Indices

•  In Progress Crew State Monitoring Research at NASA Langley

12

Task engagement has been defined in various ways. •  A subjective state defined by questionnaire factor analytic

loadings on energetic arousal (affect), task motivation, and concentration (cognition). Matthews et al. (2012)

•  A cognitive measure that increases “as a function of increasing task demands.” Task demand in the Berka et al. study was manipulated with a set of tasks that elicited the targeted cognitive states. Berka et al. (2007)

•  “a substantial amount of the variance” associated with task engagement could be explained using psychophysiological measures. Fairclough and Venables (2006)

13

The closed-loop evaluation method implies a particularly dynamic definition of task engagement •  how closely brain response slews with changes

in task demand when brain response is fed back to adjust task demand (automation/manual mix) in real time.

•  represents an “operationalization” of the task

engagement construct (Lilienfeld et al., 2015).

14

15

Biological Psychology

16

Biological Psychology

17

EEG data could be used to determine player experience across entire level designs. Nacke et.al (2010)

Engagement index (Beta / (Alpha + Theta)) was capable of differentiating high intensity game events (Player Death) from general game play. McMahan, Parberry and Parsons (2015)

Engagement Index (EI) did not show systematic vigilance decrement but discriminated cued and uncued conditions near task end. Kamzanova, Kustubayeva and Matthews (2014)

Three EEG band ratios, beta/(alpha+theta), beta/alpha, and beta/theta were able to discriminate daydreaming from attentive driving beta/alpha band ratio outperformed the other two indices. Zhao, G., Wu, C. and Ou, B (2013).

18

Adapting Automation based upon EEG Measures of Task Engagement

19

Spin-Offs from Attention and Engagement Research

20

Eastern Virginia Medical School Videogame Neurofeedback Research

21

•  HowdoesthisworkfitwithintheNASAmission?– “CongressdeclaresthatthegeneralwelfareoftheUSrequiresthattheuniquecompetenceofNASAinscienceandengineeringsystemsbedirectedtoassisFnginbioengineeringresearch,development,anddemonstra4onprogramsdesignedtoalleviateandminimizetheeffectsofdisability.”TheNa'onalAeronau'csandSpaceAct,Sec.20102.(f)

22

ZONE An embodiment of U.S. patent number 8628333

Biofeedback Training

for Optimal Athletic

Performance

23

EEG-based MindShift FPS

Trade names and trademarks are used in this presentation for identification only. Their usage does not constitute an official endorsement, either expressed or implied, by the National Aeronautics and Space Administration.

24

•  Background of Crew State Monitoring Research at NASA Langley

•  Development of a Method for Evaluating Task Engagement Indices

•  In Progress Crew State Monitoring Research at NASA Langley

25

26

Fixed-Base Simulator Lab

27

Attention-related Human Performance Limiting States (AHPLS)

•  AHPLS may reduce pilot aircraft state awareness, and can be indicated by covert or physiological markers of limited performance.

•  Current NASA Crew State Monitoring (CSM) studies assess the efficacy of using CSM technology as a means of detecting adverse human performance states.

28

• CAST research for safety enhancements to improve commercial pilot Training for Attention Management to address attention-related human performance limitations observed in flight incidents

• Develop methods to detect and measure attention-related human performance limiting states (AHPLS):

o Channelized Attention o Diverted Attention o Startle / Surprise o Confirmation Bias

29

Results are to be presented: Harrivel, Liles, Stephens, Ellis, Prinzel, Pope.

Psychophysiological sensing and state classification for attention management in commercial aviation.

AIAA Sci Tech 2016, San Diego, 4 - 8 January 2016.

30

LaRC Cockpit Motion Facility

31

Next CSM Research Objective Is Analogous to the Biofeedback Spin-Offs – “Spin-Back”

•  Commercial Aviation Safety Team (CAST) recommendation to study Training for Attention Management

•  “Training-based mitigations - self-diagnosis methods for flight crew members to recognize and recover from channelized attention, confirmation bias, startle/surprise, and diverted attention”

•  Analogous to training to recognize symptoms of hypoxia

32

For coverage of our core research: Stephens, C. L., Scerbo, M. W., and Pope, A.T. Adaptive Automation for Mitigation of Hazardous States of Awareness Chapter 26 in The Handbook of Operator Fatigue edited by Matthews, Desmond, Neubauer, and Hancock, Ashgate 2012. For coverage of our biofeedback work: Pope, A.T., Stephens, C.L., and Gilleade, K. M. Biocybernetic Adaptation as Biofeedback Training Method Chapter 5 in Advances in Physiological Computing edited by Fairclough and Gilleade, Springer 2014.

33