cognitive fidelity as a touchstone for on-demand team...

14th BRIMS Conference16-19 May 2005Copyright 1998 Institute for Simulation & Training

Cognitive Fidelity as a Touchstone for On-Demand Team Training

Benjamin BellCHI Systems, Inc.

[email protected]


Common View of “Fidelity”• High-fidelity Sims: Physical/Visual Fidelity

– wide field-of-view projection– detailed terrain– actual GFE flight controls/instrument panels– realistic sensors, systems, threats – User acceptance

• Pragmatic trade-offs…– costly to acquire and maintain– require dedicated hardware and personnel– limited supply– users can train only in specific locales during assigned time slots

• And Cognitive trade-offs– Simulators, not Trainers– requires human instructors every step of the way

These limitations are not unique to Hi-Fi Sims…but their other strengths tend to mask these limitations


On-Demand Team Training

• Need deployable training for forward/embarked personnel– 24/7 access to low-cost, low-footprint COTS hardware– Need schoolhouse + just-in-time reinforcement/refresher training

• Dimensions of On-demand Training– Anytime, anywhere– With or without the instructor– With or without the team!

• Team– Taskwork training for an individual– Team training for a group of individuals– Team skills training for an individual (w/o the team)

• Training (Simulation + assessment, feedback, debrief)


Building the Well-Tempered Trainer

• Airmanship/Stickwork Physical Fidelity• Decision-making Cognitive Fidelity • Team Coordination Social Fidelity• Communication Dialogue Fidelity

• Summary: High-fidelity simulators– wide field-of-view, detailed terrain, GFE flight controls/instrument panels– realistic sensors, systems, threats – costly to acquire/operate; require dedicated hardware and personnel– limited supply; users can train only in specific locales & assigned times

• What are the training needs: Simulators or Trainers?– Scenarios & measures tailored to training objectives?– Performance Measurement, Mission Brief and Debrief tools?– Accommodation for Team Training?

• What is the right fidelity?


The Cognitive Fidelity Gap• Communication, coordination, decision-making

– users should be immersed in environments that elicit decision-making & team behaviors that closely match the mental processes applied in practice

– implies a strong need for cognitive fidelity• Cognitive & physical fidelity not exclusive

– in fact, they are complementary– though cognitive fidelity the usual loser in spending


Bridging the Gap with Mindshare• Two factors to overcome1. Money

– Simulation-based training funding mostly acquisition– Cognitive fidelity, while comparatively less costly, has smaller $ pool– Need products/technologies that can “cross the chasm” into transition

2. Perception, or “mindshare”. – Educating end-users on utility/value of cognitive fidelity is challenging – Discussions juxtaposed in parallel with high physical fidelity simulators – Need to sway decision-makers/end-users with high-production value


Bridging the Gap with Cognitive Agents• Balanced assessment of training needs and fidelity

– Don’t contrive PC training where hi-fidelity is needed, AND…– Don’t invent reasons for hi-fidelity where not required

• Develop adaptive, instructor-optional training– Tailor scenarios to address most urgent training objectives– Dynamically generate mission brief, assessment, debrief

• Fill missing roles w/speech-capable cognitive agents– Distributed, linked training with multiple users– Synthetic teammates to fill in for missing roles– Speech recognition & synthesis in verbally-rich team settings


Cognitive Fidelity needs agents w/gravitas• Can’t get away w/Finite-State Machine models

– OK for limited, scripted behaviors (CGFs in SAF exercises)– Task/Frame agents cannot model real-world complexities

• For simulation: – Relies heavily on deterministic, close-world assumptions– Cannot scalably to multiple simultaneous, interruptable tasks

• For training– Need dynamic (not deterministic) behavior– Need variable proficiency – fallibility– Need team coordination, e.g., proactive information

exchange, compensatory actions


Cognitive Agent Case Studies

• Case Study 1: VIPERS– USAF Undergraduate Pilot Training – Situation awareness training, not flying– Sponsor: AFRL/HEA

• Case Study 2: STRATA– F/A-18 Close Air Support– Emphasis on mission management, comms,

coordination; not flying skills– Sponsor: DARPA Defense Sciences Office

• Case Study 3: VECTOR– Immersive cultural awareness training– Sponsor: ARI


Case Study 1: VIPERS• UPT Flying: High-

Traffic/Stress/Workload• Need SA of traffic, conflicts,

radiocomms• Limited airplane / sim time• Some procedures better

practiced on ground• Need guided practice, with

realistic traffic & commsBreak Pt.

Touchdown

InitialInside

Downwind

Break

Final Turn

Overhead Approach

Perch Pt.

1A/C: “CS, Gear Down”


Current Simulation Devices• Operational Flight Trainer

– High-fidelity, wide field of view– Instructor-operated– Limited flexibility– Approx. 2 per site (60-80 students)

• Unit Training Device– Cockpit mockup– Procedures trainer– Students use in pairs– No visuals– Approx. 4 per UPT base

• Good for some training but not Comms, SA


Right fidelity to meet training need?• Training Goals

– Practice for voice comms (hearing/talking)– Build pattern Situational Awareness (SA)– Not a flying skills trainer

• To achieve the right fidelity, students “chair fly”


Right-sizing Fidelity in VIPERSVirtual Interactive Pattern Environment & Radiocomms Simulator

• Training Goals– On/Off-duty self-study on low-end PCs– Improve current self-study techniques– Practice for voice comms (hearing/talking)– Build pattern Situational Awareness (SA)

• VIPERS capabilities– Environment that simulates pattern & comms– Automated assessment and feedback– Cognitive agents to role-play synthetic teammates

• RSU controller, IP, other traffic– Able to assess Comms & SA


Cognitive Models• Teammate needs: robust, realistic, interactive, fallible• Tutoring needs: measures, assessment, de-brief/AAR• Create using CHI Systems’ iGEN® cognitive agent toolkit• RSU (tower controller) model

– Interacts verbally with user– Controls all traffic in pattern behaviorally and verbally

• IP model– Verbally coaches user– Collects performance measures– Can assume control of a/c and talk to RSU

• Other traffic models– Fly a/c in the pattern– Communicate verbally over radio– Can display variable levels of proficiency


VIPERS IllustrationsPattern Orientation Missions


Case Study 2: STRATA• Synthetic Teammates for Realtime Anywhere

Training & Assessment– Focus on “headwork” – cognitive, coordination/not flying

• Cognitive Agents with realistic behaviors– Speech interactive– FAC team, Wingman

• Cognitive Fidelity Requirements– Coordinating w/Wingman: check-in, ingress/egress, alerts– CAS procedures w/FAC: brief, target ID, release params


Cognitive Fidelity: FAC Interactions• Need FAC model that

– Correctly conducts CAS strike for F/A-18 user(s)• Manages check-in, FAC-to-fighter brief• Clears user to attack only if appropriately positioned• Gives a talk-on if user fails to visually ID target• Reports BDA

– Accommodates syntactic variation across users– Recognizes incorrect actions and comms


FAC ExamplesChieftain, Hammer 21…Hammer 21, Chieftain, proceed to control point Chevy at angles 10 and contact terminal controller callsign Lightning on button redHammer 21, copy, proceeding to CP Chevy and contacting Lightning on green.Hammer 21, negative, contact terminal controller callsign Lightning on button redHammer 21, copy, proceeding to CP Chevy and contacting Lightning on red

Hammer 21, Lightning, do you have target area in sight?Hammer 21, affirmative, I have the target area in sightHammer 21, do you see the set of four hangars at the northern edge of the aircraft ramp call contactContactHammer 21, your target is the 747 SE of that groupTally target


FAC Examples

H: Lightning, Hammer 21L: Hammer 21, authenticate Alpha Romeo UniformH: Hammer 21 authenticate JulietL: Hammer 21, authenticate Alpha Mike ZuluH: Hammer 21, authenticate QuebecL: Hammer 21, Lightning, return to base.

H: Tally targetH: In wings levelL: Hammer 21, abort, return to base, you are

not on the correct attack heading, heading IP to target is two five three


Case Study 3: VECTOR• Virtual Environment Cultural Training for Operational

Readiness • Train soldiers to understand cultural differences and

develop appropriate interaction strategies• Approach

– Build engaging 3-D environment w/COTS game engine– Integrate cognitive models (iGEN®) to drive behaviors of

non-player characters (NPCs) that represent local population


Cognitive Fidelity in NPCs• Primary Goal: Create synthetic actors that can interact

w/human trainee in a cognitively realistic way• Synthetic actor requirements

– Utilize computational emotion model– Generate verbal utterances – Perceive verbal utterances from other agent and users– Physically navigate through virtual environment – Respond to trainee actions based on emotional state

• Secondary Goal: Create synthetic instructor to assess performance and deliver feedback

• Synthetic instructor requirements– Embody both cultural and pedagogical expertise– Provide real-time and post-exercise feedback– Follow trainee and “course-correct” as needed– Offer cultural advice if queried


NPC Affective Modeling• VECTOR emotion modeling focus –

– computational implementation of a simplified cognitive-appraisal model– not complex, validated emotional models (but iGEN® supports that)

• Applying ‘cognitive appraisal’ to moderate behaviors– NPC action selection– Dialog interactions

• Emotion model based on “OCC”† model of emotion– Emotional reactions include valence (pos/neg) & magnitude– VECTOR focus on implementation of 7 emotional variables

• Happiness, sadness, anger, fear, disgust, surprise, trust• Emotion model can be extended to include

– Excitatory and inhibitory effects between emotional variables – Saturation levels and a decay curve


NPC Emotion - Trusting


VECTOR Interface


Scenario Location – Clinic


VECTOR Illustration


Conclusion: Cognitive Fidelity…• Matching the right tool to the right job

– Airmanship, stick & throttle => greater physical fidelity– Training SA and decision making => greater cognitive fidelity– Practicing team coordination => synthetic teammates– Practicing comms => speech-interactive teammates

• Cognitive Fidelity not an alternative to Physical/Visual– Complementary– Cognitive agents work as well in High Fidelity as in Low

• Case studies show some important dimensions to CF– Cognitive Fidelity is not for everyone– Requires high-value interaction, speech dialogue capabilities

agent


Questions

cognitive fidelity as a touchstone for on-demand team...

Documents