Improving Collaboration in Unmanned Aerial Vehicle OperationsMarch 27, 2007

Stacey D. ScottHumans & Automation LabMIT Aeronautics and Astronauticshttp://halab.mit.edu

Teamwork in Unmanned Aerial Vehicle (UAV) OperationsIntelligence Consumers (e.g., Ground Troops)UAV OperatorsLawyers & PoliticiansGround Crew

Supporting Futuristic UAV Teams Engaged in Time-Critical OperationsFuture UAV Teams

Current Collaboration Tools Insufficient for Time-Critical OperationsIncreasing reliance on collaboration toolse.g., email, instant messaging (chat)These tools introduce communication overhead, hindering management of tasking & team coordination (Scott et al., 2006)Trend in corporate and educational collaboration systems towards intelligently sharing activity information to improve planning & coordination in networked teamsActivity awareness design approach (Carroll et al., 2006)Promoting activity awareness in UAV operator interfaces may:help UAV teams coordinate their actionsreduce the costs of obtaining activity-related information

Research Goals: Supporting Activity Awareness in Collaborative UAV OperationsTo develop and evaluate activity awareness tools for future teams of UAV operators engaged in time critical missions

To investigate the software automation and interface requirements of UAV operations team members what type of information is useful to know about ones team members and their task activities?what information do team members need to know about the UAVs activities? how does ones job duties / team role impact activity awareness requirements (e.g. mission commander, UAV operator, etc.)?

Initial Research FocusFuture UAV TeamsTechnological Support for the Team Supervisor

Assisting Team Supervisors of UAV TeamsResearch Approach: Develop representative collaborative, time-critical C2 task scenarioConduct cognitive task analysis (CTA) to generate system requirements for supervisory decision support and activity awareness displaysDevelop supervisory displays in a simulated task environmentConduct exploratory experiment

Representative Collaborative UAV Task ScenarioGround Force Protection:Futuristic UAV Operations Team performing ISR in support of a time-sensitive targeting for a ground convoy traversing a hostile region

Task Details:UAV Operations Team:1 Mission Commander3 UAV operators, each controlling multiple semi-autonomous UAVsUAVs have camera sensors onlyTeam must coordinate with external strike teamUAV team can communicate with convoy & external intelligence sources

Generating Requirements for Assisting SupervisionHybrid Cognitive Task Analysis (Nehme et al., 2006):Step 1: Scenario Task OverviewStep 2: Event Flow DiagramStep 3: Situation Awareness (SA) Requirements Step 4: Decision Ladders & Display RequirementsProduces Functional & Information Requirements

Supervisory Displays & Simulated Task EnvironmentLarge-Screen Wall DisplaysSituation Map DisplayMission Status DisplayTabletPC DisplayMission Commander DisplaySimulation & Collaboration Server (Grouplab SharedDictionary)

Designing to Promote Activity AwarenessUAV status & taskingCurrent & expected convoy safetyCurrent & expected operator task performance, relative to convoy safetySituation Map DisplayMission Status Display4(nominal)(down)81(reviewing ATR imagery)

Designing to Promote Activity AwarenessUAV status & taskingCurrent & expected convoy safetyCurrent & expected operator task performance, relative to convoy safetyPotential threat envelopeTarget strike indicatorsKnown threat envelopeSituation Map DisplayMission Status Display

Designing to Promote Activity AwarenessUAV status & taskingCurrent & expected convoy safetyCurrent & expected operator task performance, relative to convoy safetySituation Map DisplayMission Status Display

Exploratory StudySubjects: 8 participants (including 6 NROTC)Protocol:1 subject + 3 confederates per experiment2 hrs training + 2 x 15 min trialsRetrospective interview1 Subject as mCdr 3 Confederates as UAV OperatorsSituation Map DisplayMission Status Display

Some Preliminary ResultsConvoy threat summary intuitive, useful for timing command decisionsParticularly, visualizing relationship between strike team activity and convoys progress

UAV status & tasking useful for deciding when to assist UAV operatorsHowever, lack of time-on-task cues required mental effort

Operator performance displays rarely used, since surveillance progress was easily obtained from the Map481(nominal)(down)(reviewing ATR imagery)

Ongoing Work: Test-bed for Exploring Other Collaborative TST IssuesUAV Operations Team1. Interruption recovery tools for the team supervisor

Ongoing Work: Test-bed for Exploring Other Collaborative TST IssuesUAV Operations Team2. Activity awareness interfaces for the UAV operators3. New methods for deriving collaborative system requirements1. Interruption recovery tools for the team supervisor

Expanded Simulated Task EnvironmentLarge-Screen Wall DisplaysSituation Map DisplayMission Status DisplayMission Commander Display (TabletPC)Simulation & Collaboration Server (GroupLab SharedDictionary)Remote Assistance Display

Interruption Recovery Assistance for Team SupervisorMission Commander Display (TabletPC)Interactive event timeline, containing event bookmarks

Interruption Recovery Assistance for Team Supervisor (contd)Clicking on an event bookmark highlights the location of the corresponding event on the map display, e.g., a convoy attack

Preliminary UAV Operator Display DesignsMap DisplayCommunications DisplayTasking Display: TargetID & rerouting, reassigning UAVs

Improved Methodology for Deriving Collaborative System Requirements Project Goal: To develop techniques to identify dependencies in operator decision making to understand how to assist coordination of team member tasking

Future Work: Team Performance PredictionBehavioral pattern detection and performance predictions Predictive algorithms (NN, HMM, Hybrid)Alert team supervising agent of sub-optimal cognitive strategiesStress & time pressureRapidly evolving situationExcessive workload

Future Work: Team Performance PredictionBehavioral pattern detection and performance predictions Predictive algorithms (NN, HMM, Hybrid)Alert team supervising agent of sub-optimal cognitive strategiesStress & time pressureRapidly evolving situationExcessive workload

ConclusionsInitial activity awareness designs show potential for facilitating supervisory-level decision making, and supporting teamwork in generalParticularly, designs aimed at conveying current & expected status of interdependent activitiesActivity awareness approach (i.e. intelligent sharing of activity data) appears to be an appropriate design technique for large-screen displays in time-critical C2 environmentsFurther work is needed to understand what activity information should be shared (and at what level of detail) during different UAV mission types and mission phases

Contact and Sponsor InformationContacts:Dr. Stacey Scott, sdscott@mit.eduProf. Mary (Missy) Cummings, missyc@mit.eduHumans and Automation Lab (HAL) website: http://halab.mit.edu

Thanks to project sponsor:Boeing Phantom Works

Time-Sensitive Target: a target of such high priority to friendly forces that the Joint Forces Commander designates it as requiring immediate response because it poses (or will soon pose) a danger to friendly forces, or it is a highly lucrative, fleeting target of opportunity (Joint Publication 3-60, Joint Doctrine for Targeting)

CTA led to the design of two large-screen interfaces to assist team supervisionCTA helped determine relevant activity information & level of detail at which it should be displayedPerformance analysis and predictions: focus shifts from the individual to the teamPerformance analysis and predictions: focus shifts from the individual to the team